From: kymhorsell@gmail.com   
      
   I sometimes get a bit blaze about what the AI's I tinker with can do. But it's   
   really pretty remarkable. To a non stats-head it either means nothing or is   
   akin to black magic.   
      
   AI programs can be frustrating to work with. They can figure out a lot   
   but unless the design is very very careful they will not be able to   
   explain a lot in terms any human can understand.  I can imagine that a   
   civilisation could easily dead-end itself relying too much on bad AI software.   
   Not   
   that that could every happen to us! ;)   
      
   For some months I've been tinkering with new techniques to predict the   
   ebb and flow of certain activity. A couple years ago when I started out   
   looking at this stuff I thought you might be lucky to get within an   
   order of magnitude in predicting how many reports might come in next   
   month based on whatever data you could dig up that seemed related.   
   But the latest programs far exceed those old expectations.   
      
   The latest round of models the s/w has cooked up are based on   
   "planetary" movements. It turns out the month-to-month activity can be   
   predicted very closely over at least the next  days by the   
   movements of just a few key objects in our solar system.   
      
   The best result in the latest batch -- that tries to predict in   
   advance what coming month-by-month activity will be -- is here:   
      
   .   
      
   As usual, it's like an intelligence test to dig your way through the   
   results. But we all like a knotty after-dinner puzzle, don't we?   
      
   The model uses the "phase" of 8 asteroids/comets from data uploaded   
   from the good old Horizons system at JPL. The phase is roughly the   
   angle as seen from some object in the solar system between the Earth   
   and the Sun.  Back here on Earth that relates to how much of the   
   object looks lit by the sun as viewed from the ground here.   
      
   But it turns out to have other uses. E.g. in planning an   
   interplanetary trip between there and here.   
      
   The model, above, is based on a variation of the phase angle for the 8   
   objects in the model -- the "Phase Angle Bisector" (PAB).  Head   
   spinning yet? Wait. The phase is the angle as seen from some other   
   planet between the Earth and Sun at a particular date.  The *bisector*   
   is the direction exactly between the Earth and the Sun as viewed by   
   that other planet. Roughly, very roughly, it's the direction you have   
   to go to get from there to here in a vaguely reasonable time.   
      
   Not at all funny that these PAB numbers predict UFO sightings to a   
   high degree.   
      
   The list of objects in the model(s) is not particularly important.  It   
   turns out you can take almost any set of the known ~2 million objects   
   being tracked and get the same kind of model. The one listed above is   
   just the best one found in the latest twiddling.   
      
   It's hard to describe how well the model works. To number-heads we   
   look at the output from the program. For normal people they might   
   relate to the graph of the predictions and original data a bit better.   
   That's here:   
      
   .   
      
   The plot shows in green the monthly "activity" numbers between 2006   
   and 2023.  The data comes from the NUFORC. The s/w massages the   
   numbers a bit to make the model-fitting a bit more precise. We know   
   the number of sightings in a month is only a rough estimate of   
   "activity". Someone may forget exactly when they saw something and   
   just guess at the date. Some of these reports come in months or years   
   after the events. Something about stigma.   
      
   So the numbers represented by the green bars are smoothed estimates of   
   the annual rate of activity based on a given month's number of   
   sightings reports as at the start of 2023.   
      
   The dark blue line in the plot is the model estimate. You can see the model   
   is a bit of a wavy line. Since the PAB numbers from orbiting planets   
   go up and down, adding a whole bunch of them together will have this   
   appearance. Which is great because we can see activity goes up and   
   down in cycles, too.   
      
   Now here comes the knock-your-socks-off moment.   
      
   Because we have 2 million objects to choose from you might think it   
   was a slam-dunk that you could find a bunch of them that closely   
   parallels anything at all, even random sightings reports sent in by a   
   bunch of pilots and ordinary citizens looking up from their back   
   porches on a weekend evening.   
      
   And that is perfectly true.   
      
   But data science takes that into account by "information hiding".  We   
   give the stats program only PART of the data. It figures out the   
   pattern from e.g. 1/2 or 1/3 of the available data. Then we TEST the   
   model the program has found by seeing how well it matched the 50-67%   
   of the data we DID NOT GIVE IT.   
      
   It's like showing the program the back of an elephant and seeing   
   whether it can figure out the front of the elephant must have a trunk.   
      
   The spooky part of data science is -- current methods can do it!  The plot,   
   above, shows a thick blue line on the left 1/2 along   
   the bottom of the X axis. This is the part of the data the program was   
   given to predict the other 1/2 of the data. And the match is near perfect.   
      
   From a few wavy lines on the left 1/2 (the "bum" end of the elephant) it   
   predicted the "pandemic peak" on the right half. There's your trunk!   
      
   This is the difference between "curve fitting" -- a thing still   
   beloved of traditional scientists and wanna-be scientists -- and   
   "machine learning". In curve fitting the model you come up with has   
   only remembered the numbers you gave it. It has not "generalised" the   
   data to figure out what could be true in slightly different   
   circumstances.  In ML the model has "digested" the data it was given   
   and found out the rules used to generate it, and can then go off to   
   slightly or even moderately different circumstances and make a   
   reasonable prediction about what will happen.   
      
   From the bum-end of an   
   elephant it can draw a picture of the trunk without having see one before.   
      
   Spooky stuff. But just a part of what AI can do. And probably only the   
   tip of the iceberg when it comes to what Our Friends From Frolix 9 can do.   
   Predict tomrrow's lottery numbers? Not a problem. We are already building   
   machines that can do that, if you just ask them nice. :)   
      
   --   
   "Nothing in life is to be feared, it is only to be understood.   
   Now is the time to understand more, so that we may fear less."   
   - Marie Curie   
      
   The most extreme life-forms in the universe   
   New Scientist, 26 June 2008   
   There's hardly a niche on Earth that hasn't been colonised. Life can be   
   found in scalding, acidic hot pools, in the driest deserts, and in ...   
      
   [continued in next message]   
      
   --- SoupGate-Win32 v1.05   
    * Origin: you cannot sedate... all the things you hate (1:229/2)