Skip to content Skip to sidebar Skip to footer

Why Am I Seeing Inexact Floating-point Results In ECMAScript / ActionScript 3?

Hey all, let's jump straight to a code sample to show how ECMAScript/JavaScript/AS3 can't do simple math right (AS3 uses a 'IEEE-754 double-precision floating-point number' for the

Solution 1:

Welcome to the wonderful world of floating point calculation accuracy. In general, floating point calculations will give you results that are very very nearly correct, but comparing outputs for absolute equality is unlikely to give you results you expect without the use of rounding functions.


Solution 2:

This is just a side effect of using floating point numbers - these are binary representations of decimal numbers, there will always be some approximations.

Long explanation


Solution 3:

Floating point inconsistencies are a known problem in many languages. This is because computers aren't designed to handle floating point numbers.

Have fun


Solution 4:

As moonshadow states, you're running into issues with floating point precision. Floating point numbers aren't suited to the task of representing and performing arithmetic upon decimal values in the manner that you would expect. These kinds of problems are seen most often when people try to using floating point numbers for financial calculations. The wikipedia entry is good, but you might get more out of this page, which steps through an error-prone financial calculation: http://c2.com/cgi/wiki?FloatingPointCurrency

To accurately deal with decimal numbers you need a decimal library. I've outlined two BigDecimal-style libraries written in javascript that may suit your needs in another SO post, hopefully you'll find them useful:

https://stackoverflow.com/questions/744099/javascript-bigdecimal-library/1575569


Post a Comment for "Why Am I Seeing Inexact Floating-point Results In ECMAScript / ActionScript 3?"