XPost: alt.global-warming   
      
   EXECUTIVE SUMMARY:   
   - Measure twice and cut once says the old saw. According to your   
     highschool math class by averaging N repeated measurements together   
     you can reduce the error by a factor of 1/sqrt(N).  Hillbillies   
     generally say the error in the average temperature if N measurements   
     is still +1C because each thermometer has (for argument's sake) that error.   
   - But the math formula assumes the measurements are "independent" and   
     "unbiased".   
   - One web page tries to average up available met stations to arrive at   
     an average temp of the earth. The page pointedly maintains it uses   
     unprocessed temperature data in deg C.  Unfortunately the siting of   
     met stations is not "random".  They mostly cluster together around   
     population centers.  They also are not very complete. Even (say)   
     16000 met stations represent only around 1% of the land's surface,   
     and much less the total surface of the earth.   
   - We conduct an experiment with 1000s of met stations from the GHCN v3   
     dataset. Using larger and larger samples from the TAVG for a given   
     date we determine how far the sample average is from the average of   
     all stations for that date. We want to know whether the error is   
     reducing according to 1/sqrt(N) or maybe is much worse.   
   - It turns out 80% of the results obtained showed an error of +-8C. It   
     seems as samples got larger and larger the error rate "got stuck"   
     and samples larger than ~20 made no improvement to the error bounds   
     of the average of the TAVG's.   
   - A power law was fit to the observed trend and showed the error   
     (ignoring it seemed to get stuck at 8) *seemed* to follow 1/N^0.1 --   
     a much slower rate of improvement than 1/sqrt(N).  Using that   
     formula showed the error for an average of 6000 or 16000 stations   
     would still be more than +-5C.   
   - It seems the "add it up and average" of clustered met stations that   
     anyway represent only 1% of land area directly (i.e.  assuming each   
     met station faithfully represents the temperature of the surrounding   
     100 km2) is just junk science.   
   - By dividing the oceans up into e.g. 2x2 km grids and measuring the   
     IR from each at nighttime and averaging them all up for a 24 hr   
     period estimates the average temp of the oceans on the same date as the   
     dumb method but got a result of 15.85C with a claimed 3 digits of accuracy.   
     Not surprisingly that compared favorably with other published results.   
   				   
      
   For non-math types it's sometimes hard to understand that scientists   
   can calculate the warming trend attributed to AGW without actually   
   bothering to calculate an explicit "average temperature of the earth".   
      
   To many people it seems impossible to calculate rate of increase in X   
   without calculating X's and looking at how they change over time.   
      
   The resolution of the conundrum involves only high school level math.   
   Each met station on the earth is liable to see the effect of global   
   warming. If look at a number of them and "average the trends" you tend   
   to zero in on a very good estimate of the overall trend warming for AGW.   
      
   It's also true that each met station also is an approximation of the   
   average temperature of the earth. It's just that it's a very noisy   
   estimate. Adding up noisy estimates of something without careful   
   handling just adds up the noise and eventually swamps the underlying signal.   
      
   This is a useful trick lie bloggers and others employed by PR agencies   
   to fuzz up the public's understanding of what's going on with anything   
   that might be subject to product liability claims. They add up   
   disparate numbers and magnify the noise and stand back saying --   
   "look, nothing to see here".   
      
   In the case of the "average temperature of the earth" -- that some   
   climate denier sites tend to point to as evidence the earth is not   
   warming and there are no more tornadies, floods, hurricanes and   
   heatwaves like the newspapers and scientists is sayin -- the dumb way   
   of calculating this magic number is guaranteed to be way off.  Usually   
   they explicitly say they are using "unprocessed temperature data".   
      
   And it's amazing how bad this method is. It's really really laughable.   
   but as with most science, you have to have an IQ over 90 or have   
   graduated highschool to get the joke.   
      
   Standard highschool math (I met this in year 11) shows you how to   
   calculate the uncertainty in the calculation of an average of some numbers.   
      
   They say if you measure twice you need to cut only once. In some way   
   averaging 2 measurements is "better" than just one. But how much better?   
      
   The highschool math shows you that if you measure something N times   
   each with an "accuracy" of S then the error in the average of the   
   measurements is S/sqrt(N). if you take 2 measurements and average then   
   the error is reduced to 1/sqrt(2) -- about .70, i.e. about a 30%   
   improvement on 1 measurement. If you measure it 9 times and average   
   then then error is reduced by almost 70%.   
      
   But this formula relies on some assumptions. And the most important is   
   that the measurements are independent of one another. I.e.  if you're   
   cutting a length of timber you really should stand up, walk around,   
   come back and measure it again. Not just stand there and look at the   
   tape measure twice.   
      
   To measure the temperature of the earth -- if you wanted to do that --   
   you would need to measure the temperature of random spots on the   
   planet at more or less the same time. If each thermometer had an   
   accuracy of (say) 0.1C then taken 100 "independent" measurements would   
   reduce the error to 1/sqrt(100) i.e. 1/10 of .1 == .01C.   
      
   This was the basic error in the old climate denier claim you cant   
   measure any temperature better than +-1C. I remember some guy -- let's   
   call him Brent -- arguing about this basic highschool math for months   
   before someone must have taken him aside and showed him a school   
   textbook. He then deleted all those posts he's made because it seemed   
   at the time he was a college tutor.   
      
   But if the temperature measurements are not carefully selected to be   
   random then the error does not reduce as expected. It can even grow.   
      
   So this takes up so the exercise of calculating the "average   
   temperature of the earth" by summing up the "unprocessed temperature   
   measurements" from 1000s of (we assume) almost exclusively land-based   
   stations at some point in time.   
      
   We don't expect temp stations to be sited randomly all over the earth.   
   They are usually, e.g., near population centers and they -- fore sure   
   -- cluster around latitude 45N.   
      
   If you ask only registered voters for one political party their   
   voting intentions and averaging them out you will not predict the   
   outcome of too many elections.   
      
   But it is incredible just how bad this average of all stations without   
   twiddling is.   
      
   I did an experiment with the GHCN v3 stations. There are 100k of them   
   in the database, but at any one time only around 7k of them are   
   active.  Picking a random 20th cent date in August I found about 2k of   
   them had a TAVG measurement for that date.   
      
      
   [continued in next message]   
      
   --- SoupGate-Win32 v1.05   
    * Origin: you cannot sedate... all the things you hate (1:229/2)