home bbs files messages ]

Forums before death by AOL, social media and spammers... "We can't have nice things"

   comp.lang.visual.basic      MS Visual Basic discussions, NOT dot-net      10,840 messages   

[   << oldest   |   < older   |   list   |   newer >   |   newest >>   ]

   Message 9,461 of 10,840   
   Steve Gerrard to Rick Rothstein   
   Re: Very strange thing in VB6   
   01 May 05 16:36:17   
   
   From: mynamehere@comcast.net   
      
   "Rick Rothstein"  wrote in message   
   news:MaOdnckqkrlQmujfRVn-vw@comcast.com...   
      
   > Now I'm not 100% sure if the 16th place is lost during the   
   > assignment, or if it is retained but ignored every time a value is   
   > retrieved from a variable of type or sub-type Double, but the net effect   
   > is the calculation 5*1.2 becomes the value of 6, exactly, when stored in   
   > a Double type variable. You can see this like so...   
   >   
      
   I think it is the case that the 16th place is lost in the assignment, but only   
   if the result can't retain that precision. If you do   
      
       dim dblResult As Double  ' I refuse to use lng here :)   
      
       dblResult = 5 * 1.2 - 6   
       Debug.Print dblResult   
      
   dblResult will still have that check bit quantity, i.e.   
   e  -2.22044604925031E-16.   
      
   If you do this loop, you can then see that the variable loses the ability to   
   retain the "dribble" when it gets up to 3, presumably because the exponent has   
   to become larger.   
      
   Private Sub Command1_Click()   
       Dim dblResult As Double   
       Dim i As Long   
      
       dblResult = 5 * 1.2 - 6   
       Debug.Print dblResult   
      
       For i = 1 To 4   
           dblResult = dblResult + i   
           dblResult = dblResult - i   
           Debug.Print i, dblResult   
       Next i   
      
   End Sub   
      
   'result:   
   -2.22044604925031E-16   
    1            -2.22044604925031E-16   
    2            -2.22044604925031E-16   
    3             0   
    4             0   
      
   >Floating point numbers can be a real pain in the ass   
   > to work with.   
      
   Oh yes indeed :)   
      
   --- SoupGate-Win32 v1.05   
    * Origin: you cannot sedate... all the things you hate (1:229/2)   

[   << oldest   |   < older   |   list   |   newer >   |   newest >>   ]


(c) 1994,  bbs@darkrealms.ca