From: anton@mips.complang.tuwien.ac.at   
      
   Paul Rubin  writes:   
   >I'm playing with the idea of writing a Roff-like text formatter in   
   >Forth.  The input is lines of text "blah blech, and this that the   
   >other...".  The text lines can be arbitrarily long so I don't want to   
   >read the entire line into a memory buffer using something like REFILL.   
      
   Why not?  For something like Roff (or TeX or Markdown etc.) the whole   
   input file easily fits into RAM, so a line would fit, too.  The   
   question is if the Forth system supports long lines in REFILL.   
      
   To test this, I wrote the following program   
      
   .( : x dup . cr source nip <> if ." wrong length: " source . drop bye then ; )   
   cr   
      
   : gen ( n -- )   
       60 swap 0 ?do   
           dup 10 + 8 .r ."  x" dup spaces cr   
           2* loop   
       drop ;   
      
   20 gen bye   
      
   And then generated a file /tmp/long-lines.4th as follows:   
      
   gforth ./gen-long-lines.4th >/tmp/long-lines.4th   
      
   The longest line is more the 31M characters long; a system that can   
   deal with that probably can deal with any line length for which it can   
   allocate memory.  Testing various systems by loading this file, I see:   
      
   Gforth, lxf, and SwiftForth load the whole file.   
      
   iforth-5.1-mini returns to the command line after printing 970,   
   without error message.  My guess is that iForth recognized that the   
   next line is too long for its input buffer and silently called the   
   command-line interpreter.   
      
   vfx64: the last line interpreted is the one with 970 characters, and   
   an error message "wrong length: 512" is shown.  So vfx64 loaded the   
   first 512 bytes of the 970-byte line, and gave it to the text   
   interpreter.  And then the interpreted program noticed that the line   
   length is wrong, printed the error message and left the Forth system.   
      
   Back to your question, we have at least three Forth systems capable of   
   REFILLing long lines.   
      
   >Also, some input lines will be formatting commands like ".i\n" (change   
   >font to italic).  Those lines should be given to the Forth text   
   >interpreter.   
      
   And here's the signficance of REFILLing.  You could pass everything to   
   the text interpreter, and install the following recognizer sequence:   
   First one that recognizes things like ".i\n", and second one that   
   recognizes everything and then processes the line as ordinary words.   
      
   An alternative solution I would use: slurp the whole file into memory,   
   then process line by line (because of your line-oriented commands): In   
   each line, check whether something like ".i\n" occurs (with the same   
   logic that the recognizer would use); if so extract that line and   
   EVALUATE it; if not, process the line as you do by default.  I expect   
   this alternative to be slightly more work, mainly because the line   
   processing is not done automatically by the text interpreter.   
      
   >I guess I could use the FILE word set to write something like getc()   
   >with its own buffering, but that seems messy.  I'm wondering if this is   
   >a common situation and there's an idiomatic solution.   
      
   No, it's not common.  If you have too little memory, the idiomatic   
   solution is to limit the line length (see VFX64, even though it has   
   enough memory).   
      
   - anton   
   --   
   M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html   
   comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html   
        New standard: https://forth-standard.org/   
   EuroForth 2025 CFP: http://www.euroforth.org/ef25/cfp.html   
   EuroForth 2025 registration: https://euro.theforth.net/   
      
   --- SoupGate-Win32 v1.05   
    * Origin: you cannot sedate... all the things you hate (1:229/2)