home bbs files messages ]

Forums before death by AOL, social media and spammers... "We can't have nice things"

   comp.lang.visual.basic      MS Visual Basic discussions, NOT dot-net      10,840 messages   

[   << oldest   |   < older   |   list   |   newer >   |   newest >>   ]

   Message 9,470 of 10,840   
   Allcomp to Steve Gerrard   
   Re: Very strange thing in VB6   
   04 May 05 16:48:32   
   
   From: marc@nospam.allcomp.be   
      
   Thank you for your answer and sorry for the late answer (I was not in   
   the office).   
      
   In fact, I use double because I take some values from a dxf (double) and   
   by luck (bad luck), the result of calculation was 1.2   
   I have already opened a lot of DXF (mode than 100 differents) and it is   
   the first time I have exactly 1.2 to multiply by 5 (the two values   
   depent on the DXF size, coordonates...)   
      
   Thank you   
   Marc Allard   
   Allcomp   
      
      
   Steve Gerrard wrote:   
   > "Rick Rothstein"  wrote in message   
   > news:MaOdnckqkrlQmujfRVn-vw@comcast.com...   
   >   
   >   
   >>Now I'm not 100% sure if the 16th place is lost during the   
   >>assignment, or if it is retained but ignored every time a value is   
   >>retrieved from a variable of type or sub-type Double, but the net effect   
   >>is the calculation 5*1.2 becomes the value of 6, exactly, when stored in   
   >>a Double type variable. You can see this like so...   
   >>   
   >   
   >   
   > I think it is the case that the 16th place is lost in the assignment, but   
   only   
   > if the result can't retain that precision. If you do   
   >   
   >     dim dblResult As Double  ' I refuse to use lng here :)   
   >   
   >     dblResult = 5 * 1.2 - 6   
   >     Debug.Print dblResult   
   >   
   > dblResult will still have that check bit quantity, i.e.   
   > e  -2.22044604925031E-16.   
   >   
   > If you do this loop, you can then see that the variable loses the ability to   
   > retain the "dribble" when it gets up to 3, presumably because the exponent   
   has   
   > to become larger.   
   >   
   > Private Sub Command1_Click()   
   >     Dim dblResult As Double   
   >     Dim i As Long   
   >   
   >     dblResult = 5 * 1.2 - 6   
   >     Debug.Print dblResult   
   >   
   >     For i = 1 To 4   
   >         dblResult = dblResult + i   
   >         dblResult = dblResult - i   
   >         Debug.Print i, dblResult   
   >     Next i   
   >   
   > End Sub   
   >   
   > 'result:   
   > -2.22044604925031E-16   
   >  1            -2.22044604925031E-16   
   >  2            -2.22044604925031E-16   
   >  3             0   
   >  4             0   
   >   
   >   
   >>Floating point numbers can be a real pain in the ass   
   >>to work with.   
   >   
   >   
   > Oh yes indeed :)   
   >   
   >   
      
   --- SoupGate-Win32 v1.05   
    * Origin: you cannot sedate... all the things you hate (1:229/2)   

[   << oldest   |   < older   |   list   |   newer >   |   newest >>   ]


(c) 1994,  bbs@darkrealms.ca