
05 Nov 2012  #1 
 Interesting Difference in Math Calculations in different programs I noticed a difference in the results of a math problem I did in 4 separate programs, and they split 2 to 2 on the result. I multiplied the following 2 numbers in Windows Calculator, a Firefox addon calculator, Libre Office Calc and MS Excel with interesting resuts : 82.19777664*33.20088775 Results Windows Calculator = 2,729.039155524212 Firefox Addon Calculator = 2729.039155524212 Libre Office Calc = 2,729.039155524210 Microsoft Excel 2010 = 2,729.039155524210 As you can see, the calculators agree on the numbers and the spreadsheet programs agree on the numbers, but as you can also see the 2 different types of programs agreed on different numbers  it appears the spreadsheets rounded the last number down to zero. I allowed for 12 decimal places in the spreadsheet programs, I even added decimal places to 14 or 15 and I got the same results  with additional trailing zero's in the additional decimal places in the spreadsheet programs. Anyone have any idea how and why these programs are handling the math this way? Thanks for any insight anyone can shed on this riddle 
My System Specs 
. 

05 Nov 2012  #2 
 Floating point numbers (whole numbers with decimals) is not something that computers are good at. And there could be a lot of reasons why they display them as such. 
My System Specs 
05 Nov 2012  #3 
 Thanks for that reply logicearth, interesting to know 
My System Specs 
. 

07 Nov 2012  #4 
 Actually computers are just as fine with floating point numbers as they are with integers. You are observing roundoff errors (last digit) which is simply due to the number of bits assigned to the floating point number. Typically 32 bit = single precision 64 bit = double precision 128 bit = quadruple precision etc. In floating point bits are assigned to the mantissa and exponent. The precision and dynamic range is quite defined. Integer operations can also have roundoff errors. 
My System Specs 
Thread Tools  
Similar help and support threads for2: Interesting Difference in Math Calculations in different programs  
Thread  Forum  
math quiz  Chillout Room  
Change File Size Calculations  General Discussion  
Microsoft: 'Do the math' before you buy a Mac  Hardware & Devices  
need help with math/finance problem  Chillout Room  
running Long calculations  General Discussion  
Math Input Panel  General Discussion  
Is there a difference between defragging programs?  Software 
Our Sites  Site Links  About Us  Find Us 
Windows 7 Forums is an independent web site and has not been authorized,
sponsored, or otherwise approved by Microsoft Corporation.
"Windows 7" and related materials are trademarks of Microsoft Corp. © Designer Media Ltd All times are GMT 5. The time now is 02:46 AM.  