On Tuesday, February 4, 2014 6:00:46 PM UTC-5, haitic...@gmail.com wrote:   
   > On Monday, February 3, 2014 1:56:15 PM UTC-5, Phil Hobbs wrote:   
   >    
   > > On 01/31/2014 01:03 PM, haiticaore2081@gmail.com wrote:   
   >    
   > >    
      
   > On a historical note, I was dismayed to see the write-up on Wikipedia   
   calling Shannon the "Father of Information Theory." No where does Leo Szilard   
   get the credit for his ground-breaking 1936 paper on entropy and information.   
   The reason's why are    
   historical, and I will state them plainly. Szilard was a buddy of Einstein.   
   The letter starting the Manhattan Project was signed by both. But at the time   
   there was a social problem in that both were Jews, and both leftist. They   
   parked Einstein at    
   Princeton, and Szilard is relatively unknown. He never got a Nobel prize for   
   the A-bomb or information theory, and was not accepted into academic   
   establishment then.   
   >    
   > Getting back to the particular formulation of "information theory," it   
   cannot, for reasons I won't belabor here, pretend to that role. But what about   
   a lesser role, as a formula governing transmission through a channel? Is that   
   science? Maybe, but it's    
   kinda narrow, and not what the theory called "information theory" promised   
   from Szilard onwards.   
      
   I read a nice biography about Leo Szilard several years ago, "Genius in the   
   Shadows".     
   A good read.   
      
   George H.   
      
      
   >    
   >    
   >    
   > >    
   >    
   > >    
   >    
   > >    
   >    
   > > In the high-SNR limit, doubling the SNR allows you to distinguish 2x    
   >    
   > >    
   >    
   > > more unique voltage levels in a given measurement, but that only gets    
   >    
   > >    
   >    
   > > you one more bit, i.e.   
   >    
   > >    
   >    
   > >    
   >    
   > >    
   >    
   > > CC ~ BW*log2(SNR).  SNR >>1   
   >    
   > >    
   >    
   > >    
   >    
   > >    
   >    
   > > So again apart from a normalization constant, the asymptotic behaviour    
   >    
   > >    
   >    
   > > is pretty intuitive.   
   >    
   > >    
   >    
   > >    
   >    
   > OK. I'm still a skeptic in the big picture tho. I hope we can resume this   
   discussion, as I have more to say about information theory and information   
   entropy - and it's short comings. The second law is a peculiar part of   
   physics, as it's basis is    
   statistical as well as observable. And it gives physics the polarity of time,   
   as all other equations go back and forth in time.   
   >    
   >    
   >    
   > I don't mean to lecture, but here are several problems with information   
   theory (IT). First, consider Paul Revere's famous signal, one by land and two   
   by sea. According to Shannon, this has an information content of one bit. But   
   it means much more in    
   the context of that war, so just considering the channel does not adequately   
   describe the information transfer.   
   >    
   > Here is another difficulty. The second law says all spontaneous processes   
   result in an increase in entropy, or disorder. Now consider a super-saturated   
   solution which spontaneously crystallizes. So we are presented with a case   
   where a crystal has more    
   entropy than a solution it came from.    
   >    
   >    
   >    
   >    
   >    
   >    
   >    
   > >    
   >    
   > > >   
   >    
   > >    
   >    
   > > > I got into these philosophical navel gazing because I thought the   
   >    
   > >    
   >    
   > > > scientific method inadequate to investigate complex phenomena like   
   >    
   > >    
   >    
   > > > the causation of cancer. Most researchers know that it has many   
   >    
   > >    
   >    
   > > > factors, but there is no simple "container" in the scientific method   
   >    
   > >    
   >    
   > > > for that. And it was immediately relevant to attempts to make an AI   
   >    
   > >    
   >    
   > > > SW for cancer preventrics and cures.   
   >    
   > >    
   >    
   > >    
   >    
   > >    
   >    
   > > The scientific method is inadequate for a lot of things, because most    
   >    
   > >    
   >    
   > > questions that we care about in real life aren't scientific questions.   
   >    
   >    
   >    
   > That of course depends on what you mean by "scientific." But I digress from   
   the IT issue.   
   >    
   > >    
   >    
   > >    
   >    
   > >    
   >    
   > > Cheers   
   >    
   > >    
   >    
   > >    
   >    
   > >    
   >    
   > > Phil Hobbs   
   >    
   > >    
   >    
   > >    
   >    
   > >    
   >    
   > > --    
   >    
   > >    
   >    
   > > Dr Philip C D Hobbs   
   >    
   > >    
   >    
   > > Principal Consultant   
   >    
   > >    
   >    
   > > ElectroOptical Innovations LLC   
   >    
   > >    
   >    
   > > Optics, Electro-optics, Photonics, Analog Electronics   
   >    
   > >    
   >    
   > >    
   >    
   > >    
   >    
   > > 160 North State Road #203   
   >    
   > >    
   >    
   > > Briarcliff Manor NY 10510   
   >    
   > >    
   >    
   > >    
   >    
   > >    
   >    
   > > hobbs at electrooptical dot net   
   >    
   > >    
   >    
   > > http://electrooptical.net   
      
   --- SoupGate-Win32 v1.05   
    * Origin: you cannot sedate... all the things you hate (1:229/2)