clicliclic@freenet.de  wrote:   
   >   
   > "Nasser M. Abbasi" schrieb:   
   > >   
   > > On 6/7/2022 1:03 AM, clicliclic@freenet.de wrote:   
   > >   
   > > > On the page <.../reports/summer_2022/indexchapter1.htm#x2-10001> in   
   > > > the table under 1.2.1 ("Time and leaf size Performance") I think you   
   > > > should document your definition of "Normalized mean" and "Normalized   
   > > > median".   
   > > > What is devided by what, and is the mean or median computed before   
   > > > or after the normalization?   
   > > >   
   > >   
   > > Sure, will add these next build.   
   > >   
   > > Mean size is the average leaf size produced by the CAS (before any   
   > > normalization). The  Normalized mean is relative to the   
   > > mean size of the optimal anti-derivative given in the input files.   
   > >   
   > > For example, if CAS has "Normalized mean" of 3, then   
   > > the mean size of its leaf is 3 times as large as   
   > > the mean size of the optimal.   
   > >   
   > > Median size is value of leaf size where half the values   
   > > are larger than this and half are smaller (before any   
   > > normalization). i.e. The Middle value.   
   > >   
   > > Similarly the "Normalized median" is relative to the median   
   > > leaf size of the optimal.   
   > >   
   > > So if a CAS has Normalized median of 1.2, then its   
   > > median is 1.2 as large as the median leaf size of the optimal.   
   > >   
   > > > Could the addition of the Sam Blake and Waldek Hebisch test files   
   > > > affect the overall performance statistics significantly?   
   > > >   
   > >   
   > > It will change statistics for some CAS'es, but probably   
   > > not too much? Files 209 and 210 combined have about   
   > > 13,500 integrals, while Rubi's test suite (files 1 .. 208)   
   > > have a total of 71,994. So this is about 18.75% increase.   
   > >   
   > > Fyi, There were separate tests done before on just file 209 and 210   
   > > alone on my page, under section "Specialized integration tests"   
   > > but now these files are combined with the main build.   
   > >   
   > > It is good to have more variations of input test files,   
   > > this insures more coverage of each CAS.   
   > >   
   >   
   > Waldek's "Yet another integration test" suite consist's of 10,335   
   > integrands obtained by differentiation of random functions. While there   
   > is a natural number of tests in the Rubi suite - something like three   
   > to five integrands for every leaf of its decision tree -, the number in   
   > a random suite could be anything. In fact, Waldek might argue that   
   > it should be increased to 71,994 integrands for the sake of fairness.   
      
   Let me explain how I arrived at that number.  When doing random   
   testing there is problem that different random sample will   
   give different result.  Accuraccy of result is proportional   
   to square root of sample size.  So to get about 1% accuracy   
   you need 10000 samples.  10335 is artifact of how I generated   
   them: I generated more and then dropped nonsense like log(0).   
      
   Now, concerning fairness note that this collection consists   
   of log-exp functions.  Also, random functions tend to have   
   relatively large derivative.  So we get relatively large   
   function to integrate giving smaller answer.  This means   
   that integrand contains a lot of information making   
   integration easier.  It could be argued that for   
   integration more interesting are cases when integrand   
   is smaller than in random cases.  Both reasons means   
   that it would be inappropriate for those integrals to   
   dominate the testsuite.   
      
   > A natural limit would be reached if nothing new were learned about an   
   > integrator's performance when the size is increased - say, if all of   
   > the error messages from failures due to various unimplemented branches   
   > of an integration algorithm have appeared three to five times already.   
   > If there are no failures at all, only three to five tests should be   
   > used to establish integrator function.   
   >   
   > Is the present size of "Yet another integration test" too large or too   
   > small for FriCAS by this measure? This could be found out with your   
   > machinery as well as on FriCAS itself.   
      
   Let me note that by design this testsuite have very small number   
   of algeberic integrals.  OTOH when exp-log function turns out   
   to be algebraic, it is usually rather complicated algebraic   
   function and small number of them is enough to hit few error   
   messages.  Let me add that _all_ incompletness messages are   
   in algebraic part.  For purely transcendental elementary   
   integrals FriCAS claim completness (full coverage) and for   
   integration in terms of special functions FriCAS may   
   return unvaluated results even if other interals in term   
   of given special function work.  So no error messages about   
   unimplemented branches of procedure handling integration   
   in terms of special functions.  Note that depending on point   
   of view, handling of special functions is between 20% and 40%   
   of FriCAS integration code.   
      
   --   
                                 Waldek Hebisch   
      
   --- SoupGate-Win32 v1.05   
    * Origin: you cannot sedate... all the things you hate (1:229/2)