XPost: comp.lang.asm.x86, comp.lang.pascal.misc, comp.lang.pl1   
   From: robert@nospicedham.prino.org   
      
   Yes, the line that has been in Terje Mathisen's email signature for about a   
   zillion years...   
      
   But how far can you actually go caching data?   
      
   As an example I will use the development of the program I use to extract all   
   kinds of weird, wonderful and mostly just totally useless statistics from the   
   data I collect when I'm hitchhiking.   
      
   Let's start at the beginning...   
      
   Sometime, like well over 20 years ago, the timestamps of the oldest files that   
   I   
   still have is 1994-03-26, I wrote a Turbo Pascal (V3.01a) program to help me to   
   process some of my hitchhike statistics. The program was tiny, just a 49kb .COM   
   file, and did what it had to do. In the years that followed it slowly grew,   
   because I added more and more tables, (Driver: "Have you ever calculated this   
   or   
   that?" Me: "No, but it might be interesting...") and the last (60th) Turbo   
   Pascal V3.01a version, with files dated @ 1996-08-05, came in at a hefty 54kb.   
   :)   
      
   As a side note, in my day-to-day job I work as a PL/I programmer on z/OS, so I   
   also translated the Pascal code to PL/I and the oldest (of currently 35)   
   versions written in that language dates back to 1995-10-22.   
      
    From version 46 (1995-07-30) the code contained lots of inline (the old type   
   inline, with hexadecimal codes) code, and from version 51 (1995-10-13) yours   
   truly started adding 0x66 (386) prefixes to it. The last TP 3.01a version (60)   
   saw the light on 1996-08-05, when ...   
      
   A fellow contractor at KLM gave me his no longer being used version of TP 6.   
      
   Given that my carefully byte counted TP 3.01a inline code sometimes crossed   
   actual Pascal code, the first TP 6 version (obviously) didn't run, and I had to   
   convert all inline code back to normal Pascal, ouch!   
      
   The last (53rd) TP 6 version saw the light on 2008-10-06, and the reason for   
   abandoning Turbo Pascal was the fact that it could no longer cope with the   
   amount of data, at the time I had hitched 2,206 rides and the input file had   
   reached a size of 502kb, way too much to handle in the 16-bit DOS environment,   
   even using using a special CONFIG.SYS menu and AUTOEXEC.BAT that left most of   
   memory free, and a unit that extended the TP heap into unused UMBs.   
      
   I had to change again, and having been a lifelong user of Pascal (and its   
   rather   
   more functional grandparent, PL/I), I choose to go for Virtual Pascal, a 32-bit   
   clone of Turbo/Borland Pascal. I did think about FreePascal at the time, and   
   actually gave it a try, but in the end (don't ask me for the then "Why?") it   
   didn't cut it, and having tried it again several times in the past few years,   
   it   
   still doesn't cut it, the problem *seems* to be that FP uses a different   
   philosophy with regards to alignment of data, which even causes the "pure"   
   Pascal version to generate code that eventually results in a GPF.   
      
   Obviously, having recovered from the TP 3.01a "inline" to TP 6 conversion   
   fiasco, I had made sure that nothing of this nature would ever occur again, by   
   keeping two parallel versions of the TP 6 source, a "pure" Pascal one, and one   
   with inline assembler, and other than having to change a few registers in one   
   not entirely "pure" Pascal routine, some integers to longint's and, although   
   this happened somewhat later and wasn't trivial, all Borland 6-byte REALs to   
   IEEE doubles, the first Virtual Pascal version was nearly identical to the last   
   TP6 version. (I'm currently working on the 93rd version coded in VP)   
      
   Besides the fact that VP is a 32-bit compiler, which means I will never run out   
   of space again, it's a real clone of the Borland software, which means that the   
   generated code isn't, to say it very politely, much better than the TP 6 (AD   
   1990) or even TP 3.01a (AD 1985) code, but unlike the Borland compilers, it   
   actually produces an assembler listing that shows the generated code and as a   
   result I've spent quite a bit of time over the last few years hacking code from   
   Pascal to inline assembler. I usually let the compiler do the heavy lifting   
   when   
   it comes to new code, but once the Pascal code is working, the routine is   
   duplicated between conditional compilation tags, and, using the original   
   assembler listing as a starting point, converted into inline assembler, which   
   in   
   many, if not most, cases now only vaguely resembles the original code, I try to   
   keep data in registers, avoid the use of the VP RTL where possible, and if ebx,   
   esi and edi, the inter-procedural to-be-saved registers aren't used in the   
   calling chain, I will remove the saving and restoring of them, or save them in   
   the calling procedure, rather than in the called one. The result of all of this   
   fiddling? A program that is around 35% faster and nearly 25% smaller that the   
   pure Pascal version... In (slightly outdated, I started writing this post in   
   December 2015) figures, with the running time averaged over 10 runs?   
      
   Inline assembler version:  76,288 bytes, running time 0:00:00.23 sec   
   Pure Pascal version     : 101,376 bytes, running time 0:00:00.35 sec   
      
   Yes, all this work for a program that runs in a fraction of a second...   
      
   So why this posting?   
      
   In the past years I've posted a few snippets of the program:   
      
   Subject: Optimization problem   
   Message-ID: <85akp6F4bpU1@mid.individual.net>   
   Date: Sun, 16 May 2010 18:28:26 +0000   
      
   Thread ran to 70 postings, and in the end I actually found the bug in the code   
   sent to me by Paul Green way back in May 1998, which does not require any   
   sorting, although it (currently) does an excessive(?) amount of allocating and   
   freeing of memory via the RTL - 858 allocates and frees at the last run of the   
   equivalent PL/I program. (This could, in theory and obviously in practice too,   
   be avoided by pre-allocating, once, an array to hold the data)   
      
   Subject: x86 to x86-64 conversion issue   
   Message-ID: <9lppukFgb0U1@mid.individual.net>   
   Date: Mon, 26 Dec 2011 01:29:22 +0000   
      
   Thread ran to 17 postings, I'm using Steven Fürst's solution, due to lack of   
   support for anything over MMX in Virtual Pascal :(   
      
   Subject: Replacing single byte in doubleword   
   Message-ID: <9v7tt9Fn4vU1@mid.individual.net>   
   Date: Wed, 18 Apr 2012 15:29:19 +0200   
      
   Tread ran to 12 postings, I'm using Terje's solution :)   
      
   So I'm now going to be a heretic, and abandon x86 in favour of PL/I, as that   
   makes it a bit easier to show the data-structures I'm working with.   
      
   When I read in the data, from a simple codepage 437 spaced-out CSV file   
   (currently just over 1Mb, and containing the data for the 3,725 recorded rides   
      
   [continued in next message]   
      
   --- SoupGate-Win32 v1.05   
    * Origin: you cannot sedate... all the things you hate (1:229/2)