Ask Me Help Desk

Ask Me Help Desk (https://www.askmehelpdesk.com/forum.php)
-   Java (https://www.askmehelpdesk.com/forumdisplay.php?f=440)
-   -   Calculation in java code (https://www.askmehelpdesk.com/showthread.php?t=329407)

  • Mar 15, 2009, 03:51 AM
    yugandhar
    Calculation in java code
    In my program total of fields is 53.25 but it is giving 53.24999

    Why it is happening like that
  • Mar 15, 2009, 03:53 AM
    yugandhar
    Quote:

    Originally Posted by yugandhar View Post
    In my program total of fields is 53.25 but it is giving 53.24999

    why it is happening like that

    :confused:
  • Mar 15, 2009, 07:54 PM
    Perito

    This happens not only in Java code, but in almost all computer code. The problem is a very common one. The problem is that it's not always possible to represent a floating-point decimal number in a binary representation. You're seeing a "round off error".

    You can take steps to minimize it, like using 8-byte floating point numbers instead of 4-byte floating-point numbers. That will help, but it may not eliminate it entirely.

    When you do comparisons, the usual "fix" is to use "not equal" instead of "equal" comparisons or to compare with a value that is slightly larger than the value you expect.
  • Mar 15, 2009, 08:26 PM
    KISS

    If you want two places and the result is positive then take the number and multiply by 100 and add 0.5. Take the FIX() of it and divide by 100

    e.g. 53.24999;
    *100 = 5324.999
    +0.5= 5325.49
    Fix() = 5325.
    /100 = 53.25

    you have to watch integer () which is the greatest integer in x function and truncation. I forget what works for negative numbers. You may have to use the sgn() function.
    Functions may not be JAVA.

    This is where formatted output is useful.
  • Mar 18, 2009, 12:44 PM
    jstrike
    Perito gave a good answer as to why it's happening.

    You can use the BigDecimal class to round the number but there's no way you can prevent it from happening. (Don't use Math.round)

  • All times are GMT -7. The time now is 03:34 AM.