Forums before death by AOL, social media and spammers... "We can't have nice things"
|    comp.databases.ms-sqlserver    |    Notorious Rube Goldberg contraption    |    19,505 messages    |
[   << oldest   |   < older   |   list   |   newer >   |   newest >>   ]
|    Message 19,372 of 19,505    |
|    Erland Sommarskog to Anton Shepelev    |
|    Re: Strage difference between precision     |
|    25 Apr 22 21:58:30    |
      From: esquel@sommarskog.se              Anton Shepelev (anton.txt@gmail.com) writes:       > Hello, all       >       > Can you please tell me why this query:       >       > SELECT POWER(0.5, 2), POWER(0.50, 2)       > Returns 0.3 0.25       > Instead of 0.25 0.25       >              All say after me: "SQL Server is designed for maximum confusion!".              I know how it feels, I've run into exactly this thing myself. And       I thought, what!? Then I read the manual and learnt this:              The retun type of power() is the type of the first argument. The 0.3       has the type decimal(1, 1), and so the return type is numeric(1,1)       and you get 0.3. Intuitive? Maybe not. But this is what the manual       says.              By the way, this is how you can tell the type of a numeric literal:       DECLARE @s sql_variant = 0.3       SELECT sql_variant_property(@s, 'Basetype'),        sql_variant_property(@s, 'Precision'),        sql_variant_property(@s, 'Scale')              To avoid such surpises, say something like:               SELECT cast(power(cast @val as float) as decimal(10,3))              --- SoupGate-Win32 v1.05        * Origin: you cannot sedate... all the things you hate (1:229/2)    |
[   << oldest   |   < older   |   list   |   newer >   |   newest >>   ]
(c) 1994, bbs@darkrealms.ca