
05 Nov 2012  #1 

Interesting Difference in Math Calculations in different programs
I noticed a difference in the results of a math problem I did in 4 separate programs, and they split 2 to 2 on the result. I multiplied the following 2 numbers in Windows Calculator, a Firefox addon calculator, Libre Office Calc and MS Excel with interesting resuts :
82.19777664*33.20088775 Results Windows Calculator = 2,729.039155524212 Firefox Addon Calculator = 2729.039155524212 Libre Office Calc = 2,729.039155524210 Microsoft Excel 2010 = 2,729.039155524210 As you can see, the calculators agree on the numbers and the spreadsheet programs agree on the numbers, but as you can also see the 2 different types of programs agreed on different numbers  it appears the spreadsheets rounded the last number down to zero. I allowed for 12 decimal places in the spreadsheet programs, I even added decimal places to 14 or 15 and I got the same results  with additional trailing zero's in the additional decimal places in the spreadsheet programs. Anyone have any idea how and why these programs are handling the math this way? Thanks for any insight anyone can shed on this riddle 
My System Specs 
. 
05 Nov 2012  #2 

Floating point numbers (whole numbers with decimals) is not something that computers are good at. And there could be a lot of reasons why they display them as such.

My System Specs 
05 Nov 2012  #3 

Thanks for that reply logicearth, interesting to know

My System Specs 
. 

07 Nov 2012  #4 

Actually computers are just as fine with floating point numbers as they are with integers. You are observing roundoff errors (last digit) which is simply due to the number of bits assigned to the floating point number. Typically
32 bit = single precision 64 bit = double precision 128 bit = quadruple precision etc. In floating point bits are assigned to the mantissa and exponent. The precision and dynamic range is quite defined. Integer operations can also have roundoff errors. 
My System Specs 
Thread Tools  Search this Thread 
Similar help and support threads  
Thread  Forum  
A math quiz You've seen a SSD for $97. Since you have no money you borrow $50 from your dad and $50 from your mom. $50 + $50 = $100 right? You buy the SSD and get $3 back. You give your dad and mom $1 each and $1 you keep. So you now owe your parents $49 each. $49 + $49 = $98 + your $1 = $99, right?... 
Chillout Room  
math quiz a boy was very naughty his teacher gave him an assignment to keep him busy that is add all the numbers from 1 to 100 in couple of minutes he came back with the answer 5,050 that is he multiplied 50 * 101 
Chillout Room  
Change File Size Calculations It appears windows does not cnform to the standard method of calculating file size. e.g. the standard value of 1GB is 1000MB, however windows uses 1024MB. As such it is not in fact a "Gigabyte" But a "Gibibyte". This is extremley annoying, as for egxample, I have a 2TB hard Drive, which in the... 
General Discussion  
Microsoft: 'Do the math' before you buy a Mac cont here 
Hardware & Devices  
running Long calculations Just upgraded to W7. This problem arises, which I didn't have in XP: I run a program (one that I've developed) that does a huge number of numerical calculations. To reach its conclusion and give the answer takes hours and hours. Best to start it before bedtime, hoping to have the answer in the... 
General Discussion  
Is there a difference between defragging programs? Hi, I was just wondering if there is a difference between disk defrag programs? Is there anyone that isnt as good as the other? I would like to know between Iobit smart defrag, Tuneup Utilities 2009 defrag, diskeeper and auslogic disk defrag. Which one is better or doesnt it make a difference?... 
Software 
Our Sites 
Site Links 
About Us 
Find Us 
Windows 7 Forums is an independent web site and has not been authorized,
sponsored, or otherwise approved by Microsoft Corporation.
"Windows 7" and related materials are trademarks of Microsoft Corp.
© Designer Media Ltd All times are GMT 5. The time now is 05:52. 
