PDA

View Full Version : Calculation in java code


yugandhar
Mar 15, 2009, 03:51 AM
In my program total of fields is 53.25 but it is giving 53.24999

Why it is happening like that

yugandhar
Mar 15, 2009, 03:53 AM
In my program total of fields is 53.25 but it is giving 53.24999

why it is happening like that
:confused:

Perito
Mar 15, 2009, 07:54 PM
This happens not only in Java code, but in almost all computer code. The problem is a very common one. The problem is that it's not always possible to represent a floating-point decimal number in a binary representation. You're seeing a "round off error".

You can take steps to minimize it, like using 8-byte floating point numbers instead of 4-byte floating-point numbers. That will help, but it may not eliminate it entirely.

When you do comparisons, the usual "fix" is to use "not equal" instead of "equal" comparisons or to compare with a value that is slightly larger than the value you expect.

KISS
Mar 15, 2009, 08:26 PM
If you want two places and the result is positive then take the number and multiply by 100 and add 0.5. Take the FIX() of it and divide by 100

e.g. 53.24999;
*100 = 5324.999
+0.5= 5325.49
Fix() = 5325.
/100 = 53.25

you have to watch integer () which is the greatest integer in x function and truncation. I forget what works for negative numbers. You may have to use the sgn() function.
Functions may not be JAVA.

This is where formatted output is useful.

jstrike
Mar 18, 2009, 12:44 PM
Perito gave a good answer as to why it's happening.

You can use the BigDecimal class to round the number but there's no way you can prevent it from happening. (Don't use Math.round)