home bbs files messages ]

Forums before death by AOL, social media and spammers... "We can't have nice things"

   comp.compilers      Compiler construction, theory, etc. (Mod      2,753 messages   

[   << oldest   |   < older   |   list   |   newer >   |   newest >>   ]

   Message 1,352 of 2,753   
   Hans Aberg to Hans-Peter Diettrich   
   Re: Tokenizer theory and practice   
   17 May 08 11:13:26   
   
   From: haberg_20080406@math.su.se   
      
   Hans-Peter Diettrich wrote:   
   > Unicode introduces a couple of problems into lexers, which I don't want   
   > to discuss too deeply. Most important seems to be the expansion of the   
   > character codes, from single to multiple bytes.   
      
   Unicode regular expressions can be lexed directly by rewriting into UTF.   
   I posted some Haskell function for doing that here   
      http://lists.gnu.org/archive/html/help-flex/2005-01/msg00043.html   
      
      Hans Aberg   
      
   --- SoupGate-Win32 v1.05   
    * Origin: you cannot sedate... all the things you hate (1:229/2)   

[   << oldest   |   < older   |   list   |   newer >   |   newest >>   ]


(c) 1994,  bbs@darkrealms.ca