Forums before death by AOL, social media and spammers... "We can't have nice things"
|    comp.lang.visual.basic    |    MS Visual Basic discussions, NOT dot-net    |    10,840 messages    |
[   << oldest   |   < older   |   list   |   newer >   |   newest >>   ]
|    Message 9,440 of 10,840    |
|    Rick Rothstein to All    |
|    Re: Very strange thing in VB6    |
|    27 Apr 05 10:05:02    |
      From: rickNOSPAMnews@NOSPAMcomcast.net              > > I have seen something really strange in VB6       > >       > > If I do a Int ( (5 * 1.2)) , I receive the value 5, but I should       receive 6?       > > Is this a bug or something really "normal".       > > I can see that if I do       > > ? int ((5 * 1.2 + 0.0000000000000003))       > > I receive 6. If I add something smaller, I have 5 as a result       > > What is strange is that If I do a ? 5*1.2, I receive 6 so (5 * 1.2)       > > should do 6       > > If I do Int ( (5 * 12 / 10)), the result is OK (6)       > > What can I do to have it correct?       > > Is there an option in VB6?       > >       > > Thank you       > > Marc Allard       > > Allcomp       > >       >       > Isn't this just typical of the inaccuracies often produced       > when using binary to manipulate decimal numbers?              Yes. See http://support.microsoft.com/default.aspx?scid=kb;en-us;42980       for more details.                     > I suspect the actual result is probably something       > like 5.999999999 (etc) and the INT of that will be 5.              If you execute this statement: Print 5 * 1.2 - 6       You will get -2.22044604925031E-16 as a response proving that 5*1.2 is       not exactly 6 when the considerations of the above article are kept in       mind.                     > Using Cint(5 * 1.2) will give the correct answer of 6 as       > Cint round fractions whereas INT does not.              The only problem with that suggestion is the CInt function uses Banker's       Rounding, so you may still get unexpected results (for a different       reason than above)...               MsgBox CInt(3 * 1.5)              displays 4, not the expected value of 5.                     Rick - MVP              --- SoupGate-Win32 v1.05        * Origin: you cannot sedate... all the things you hate (1:229/2)    |
[   << oldest   |   < older   |   list   |   newer >   |   newest >>   ]
(c) 1994, bbs@darkrealms.ca