home bbs files messages ]

Forums before death by AOL, social media and spammers... "We can't have nice things"

   comp.lang.visual.basic      MS Visual Basic discussions, NOT dot-net      10,840 messages   

[   << oldest   |   < older   |   list   |   newer >   |   newest >>   ]

   Message 9,440 of 10,840   
   Rick Rothstein to All   
   Re: Very strange thing in VB6   
   27 Apr 05 10:05:02   
   
   From: rickNOSPAMnews@NOSPAMcomcast.net   
      
   > > I have seen something really strange in VB6   
   > >   
   > > If I do a Int ( (5 * 1.2)) , I receive the value 5, but I should   
   receive 6?   
   > > Is this a bug or something really "normal".   
   > > I can see that if I do   
   > > ? int ((5 * 1.2  + 0.0000000000000003))   
   > > I receive 6. If I add something smaller, I have 5 as a result   
   > > What is strange is that If I do a ? 5*1.2, I receive 6 so (5 * 1.2)   
   > > should do 6   
   > > If I do Int ( (5 * 12 / 10)), the result is OK (6)   
   > > What can I do to have it correct?   
   > > Is there an option in VB6?   
   > >   
   > > Thank you   
   > > Marc Allard   
   > > Allcomp   
   > >   
   >   
   > Isn't this just typical of the inaccuracies often produced   
   > when using binary to manipulate decimal numbers?   
      
   Yes. See http://support.microsoft.com/default.aspx?scid=kb;en-us;42980   
   for more details.   
      
      
   > I suspect the actual result is probably something   
   > like 5.999999999 (etc) and the INT of that will be 5.   
      
   If you execute this statement: Print 5 * 1.2 - 6   
   You will get -2.22044604925031E-16 as a response proving that 5*1.2 is   
   not exactly 6 when the considerations of the above article are kept in   
   mind.   
      
      
   > Using Cint(5 * 1.2) will give the correct answer of 6 as   
   > Cint round fractions whereas INT does not.   
      
   The only problem with that suggestion is the CInt function uses Banker's   
   Rounding, so you may still get unexpected results (for a different   
   reason than above)...   
      
        MsgBox CInt(3 * 1.5)   
      
   displays 4, not the expected value of 5.   
      
      
   Rick - MVP   
      
   --- SoupGate-Win32 v1.05   
    * Origin: you cannot sedate... all the things you hate (1:229/2)   

[   << oldest   |   < older   |   list   |   newer >   |   newest >>   ]


(c) 1994,  bbs@darkrealms.ca