From: krishna.myneni@ccreweb.org   
      
   On 9/19/24 01:33, mhx wrote:   
   > On Thu, 19 Sep 2024 0:28:33 +0000, Krishna Myneni wrote:   
   >   
   >> On 9/12/24 21:10, Krishna Myneni wrote:   
   > [..]   
   >> A bit more involved test of the same 64-bit PRNGs starts to show the   
   >> possibility of defects in Marsaglia's 64-bit kiss prng (RAN-KISS)   
   >> compared to a simple 64-bit LCG prng (RANDOM).   
   > [..]   
   >   
   > I wonder if it is at all possible to really prove something   
   > about the PRNG *with tests of this type*. Intuition wants us to   
   > believe that the longer we run the simulation, the closer   
   > the result must match the expected outcome. Shouldn't we   
   > compute the probability that after a certain size run the   
   > result does NOT match the known result (given an ideal PRNG),   
   > or how unlikely it is that the result has a given error?   
   >   
      
   Perfectly valid questions, and also addressable; however, the purpose at   
   present is to *compare the relative performance* of two different prngs   
   for a simulation in which the answers are known in the limit of large N.   
   Therefore, while we haven't considered how quickly with N an ideal PRNG   
   should converge to the known , , , a comparison of the   
   relative errors of two PRNGs, at given suitably large N, should be a   
   valid comparison of the relative performance of the two PRNGs *for this   
   particular problem*.   
      
   Consider that the moments computed from the simulation describe how well   
   the histogram of the set of random samples obtained, for fixed N,   
   describe the shape of the ideal Maxwell-Boltzmann distribution in the   
   limit of large N. The main question which needs to be addressed is   
   whether or not N is sufficiently large in the tests shown above for the   
   comparison between PRNG A and PRNG B to be valid.   
      
      
   > Example: say the result of PRNG-a consistently has one of   
   > its bits (say bit 0) stuck at zero. Would the test under   
   > consideration detect this specific problem at all?   
   >   
      
   This is easy to test, for example with the LCG prng. We can modify the   
   original,   
      
   : random  ( -- u ) LCG_MUL seed @ * LCG_ADD + dup seed ! ;   
      
   as follows:   
      
   \ variation of RANDOM with stuck bit 0   
      
   -1 1 LSHIFT constant SB_MASK   
   : random-sb ( -- u_sb )   
       LCG_MUL seed @ * LCG_ADD + SB_MASK and dup seed ! ;   
      
   Now, run the test on RANDOM-SB   
      
   ' random-sb test-prng   
      
   Moments of speed   
     N        (m/s)     (m/s)^2     (m/s)^3   
   10^2     1181.0956     1656472.7       2604709063.   
   10^3     1293.3130     1952149.7       3300955817.   
   10^4     1259.3279     1862988.3       3108515117.   
   10^5     1260.5577     1872157.8       3147664636.   
   10^6     1259.4425     1868918.9       3139487337.   
   10^7     1259.6136     1869145.0       3139092438.   
      
   Relative Errors   
     N       |e1|       |e2|       |e3|   
   10^2  6.24e-02   1.14e-01   1.71e-01   
   10^3  2.67e-02   4.42e-02   5.12e-02   
   10^4  3.17e-04   3.50e-03   1.01e-02   
   10^5  6.59e-04   1.40e-03   2.39e-03   
   10^6  2.26e-04   3.31e-04   2.09e-04   
   10^7  9.04e-05   2.10e-04   3.35e-04   
      
   In general, the relative errors shown in this table are higher than for   
   the original RANDOM prng: for N=10^7, there is an increase of about a   
   factor of 10 in the relative errors for e2 and e3, and a smaller   
   increase in error for e1. Only at N = 10^6 are the errors for RANDOM-SB   
   actually smaller than for RANDOM, which suggests that we may need to   
   increase N for meaningful comparison.   
      
   --   
   Krishna   
      
   --- SoupGate-Win32 v1.05   
    * Origin: you cannot sedate... all the things you hate (1:229/2)