XPost: comp.theory   
   From: markwh04@yahoo.com   
      
   On Apr 8, 12:44 pm, Tegiri Nenashi  wrote:   
   > Formally, a set of terminals is partitioned into a separator or a set   
   > of separators, and the rest of terminals. Then, string tokenizer   
   > translates a given word into a set (or list) of words. Here we have   
   > the first technical difficulty, what exactly this translation is?   
      
   The inverse of a monoid homomorphism. The homomorphism it is the   
   inverse of is the one that maps the monoid of token sequences to the   
   monoid of character sequences.   
      
   --- SoupGate-Win32 v1.05   
    * Origin: you cannot sedate... all the things you hate (1:229/2)