Why Am I Seeing Inexact Floating-point Results In ECMAScript / ActionScript 3?
Solution 1:
Welcome to the wonderful world of floating point calculation accuracy. In general, floating point calculations will give you results that are very very nearly correct, but comparing outputs for absolute equality is unlikely to give you results you expect without the use of rounding functions.
Solution 2:
This is just a side effect of using floating point numbers - these are binary representations of decimal numbers, there will always be some approximations.
Solution 3:
Floating point inconsistencies are a known problem in many languages. This is because computers aren't designed to handle floating point numbers.
Solution 4:
As moonshadow states, you're running into issues with floating point precision. Floating point numbers aren't suited to the task of representing and performing arithmetic upon decimal values in the manner that you would expect. These kinds of problems are seen most often when people try to using floating point numbers for financial calculations. The wikipedia entry is good, but you might get more out of this page, which steps through an error-prone financial calculation: http://c2.com/cgi/wiki?FloatingPointCurrency
To accurately deal with decimal numbers you need a decimal library. I've outlined two BigDecimal-style libraries written in javascript that may suit your needs in another SO post, hopefully you'll find them useful:
https://stackoverflow.com/questions/744099/javascript-bigdecimal-library/1575569
Post a Comment for "Why Am I Seeing Inexact Floating-point Results In ECMAScript / ActionScript 3?"