From: tr.17687@z991.linuxsc.com   
      
   antispam@fricas.org (Waldek Hebisch) writes:   
      
   > Michael S  wrote:   
   >   
   >> I experimented a bit more (in fact, more like a lot more) with   
   >> test batteries of L?Ecuyer.  It led me to conclusion that occasional   
   >> failure in the either middle or big battery means nothing.   
   >> Sometimes even cripto-quality PRNG does not pass one or another test.   
   >> Then you try to reproduce it and see that with any other seed that you   
   >> try a failure does not happen.   
   >> All in all, it makes me more suspect of PRNGs that consistently pass   
   >> both batteries with various seed.  I start to see it as a sign of   
   >> PRNG being rigged to pass tests.   
   >   
   > Well, that depends on the tests and threshhold in the tests.   
   > Some tests when fed with trurly random source will produce produce   
   > very small variation of the results.  With generous threshhold   
   > such test will essentially never fail for trurly random source.   
   > OTOH when expected variation of the results is larger and   
   > threshhold is tight, then trurly random source will fail the   
   > test from time to time.  And if you test long enough you should   
   > be able to estimate probability of failure and possibly compare   
   > is with theoretical result if available.   
      
   It's inherent in the nature of statistical testing that every   
   so often a statistical test will "fail" even for a truly random   
   input.  An input source that never fails can also be indicative   
   of a low quality source (and perhaps one that was tuned to the   
   particular set of tests being done).  It took me a while to   
   learn that the results of a PRNG test suite should not be seen   
   as purely binary.   
      
   --- SoupGate-Win32 v1.05   
    * Origin: you cannot sedate... all the things you hate (1:229/2)