Windows 7 Forums

Welcome to Windows 7 Forums. Our forum is dedicated to helping you find support and solutions for any problems regarding your Windows 7 PC be it Dell, HP, Acer, Asus or a custom build. We also provide an extensive Windows 7 tutorial section that covers a wide range of tips and tricks.


Windows 7: Lies, damned lies, and benchmarks: is IE9 cheating at SunSpider?


20 Nov 2010   #1
JMH

Win 7 Ultimate 64-bit. SP1.
 
 
Lies, damned lies, and benchmarks: is IE9 cheating at SunSpider?

Quote:

The SunSpider JavaScript performance benchmark, devised by the developers of the WebKit browser engine, is used and quoted widely as a measure of browser scripting performance. A surprising result was recently noticed by a Mozilla developer, Rob Sayre, looking at Internet Explorer 9's performance in this test. On one of the many subtests it performs, Internet Explorer 9 was finishing the subtest almost instantly.

In and of itself, that's not necessarily very interesting; several of the subtests in SunSpider are near-instant in the browser. However, it piqued the developer's curiosity. He made some minor changes to the test—changes that don't alter the result of the calculation the test performs and that, naively at least, should be treated as equivalent—and saw Internet Explorer 9 slow down considerably. He filed a bug against Internet Explorer.

Sayre's bug report was conservative—he suggested that an optimization that Internet Explorer 9's Chakra JavaScript engine was performing was fragile, and was easily disabled by minor alterations to the code that it should disregard. Coverage earlier today of the same issue was less guarded: Internet Explorer 9 was accused of cheating in the test. The allegation is that Microsoft has built a specific optimization into Chakra that detects, and bypasses, the specific code in SunSpider, but which has no other purpose. In other words, the optimization will not do anything to improve the browser's performance in any other scenario.
Lies, damned lies, and benchmarks: is IE9 cheating at SunSpider?


My System SpecsSystem Spec
.

21 Nov 2010   #2

Windows 7 Professional x64
 
 

I've read about this a couple of days ago. Microsoft isn't really cheating. JavaScript engine in IE9 is just a bit more intelligent, in that it can detect useless code and skip executing it. It's called dead code optimization and the article explains it. In IE9's case, the JavaScript engine has been optimized to do very well in the SunSpider JavaScript benchmarks.

The real question here is that since it is clearly shown that a browser can be tweaked to do well in certain benchmarks, how reliable are benchmarks?
My System SpecsSystem Spec
21 Nov 2010   #3

Windows 8 Pro x64
 
 

I use real situations for testing, and such testing has shown Google Chrome to be the fastest for the sites I visit.
My System SpecsSystem Spec
.


22 Nov 2010   #4
Lee

Win 7 Pro x64, VM Win XP, Win7 Pro Sandbox, Kubuntu 11
 
 

Who really cares? This is just plain silly. Oh Teacher, Steve is cheating again. As if no one else does it. Oh darn, but Google doesn't, and I heard that Apple is always telling the truth.

Get over it, they all have their own way of testing; this helps them look good in the eyes of those who are not technically inclined. Beside once someone has chosen a application or browser it is what they have a tendency to stay with.
My System SpecsSystem Spec
22 Nov 2010   #5

Systems 1 and 2: Windows 7 Enterprise x64, Win 8 Developer
 
 

Video cards did (and still do) optimize for benchmark tests. They're playing the same old song. It's not that big of a deal with browsers, as no expense is laid out. It's rougher with video cards, as you buy it, you own it.
My System SpecsSystem Spec
22 Nov 2010   #6

Win7 x64 Ultimate SP1
 
 

Quote   Quote: Originally Posted by Lee View Post
Who really cares? This is just plain silly. Oh Teacher, Steve is cheating again. As if no one else does it. Oh darn, but Google doesn't, and I heard that Apple is always telling the truth.

Get over it, they all have their own way of testing; this helps them look good in the eyes of those who are not technically inclined. Beside once someone has chosen a application or browser it is what they have a tendency to stay with.
Exactly
My System SpecsSystem Spec
22 Nov 2010   #7

Systems 1 and 2: Windows 7 Enterprise x64, Win 8 Developer
 
 

Quote   Quote: Originally Posted by ken9122 View Post
Quote   Quote: Originally Posted by Lee View Post
Who really cares? This is just plain silly. Oh Teacher, Steve is cheating again. As if no one else does it. Oh darn, but Google doesn't, and I heard that Apple is always telling the truth.

Get over it, they all have their own way of testing; this helps them look good in the eyes of those who are not technically inclined. Beside once someone has chosen a application or browser it is what they have a tendency to stay with.
Exactly
We do tend to fall in love with our browsers. And, speaking as one, I like to see it do well in the benchmarks. If it becomes painfully obvious it is lacking, I would consider switching. Different browsers, different search engines, different platforms. Ain't computing great!
My System SpecsSystem Spec
22 Nov 2010   #8

ultimate 64 sp1
 
 

Quote:
when what's important is not measurable, what's measurable becomes important.
they all lie.
My System SpecsSystem Spec
22 Nov 2010   #9

Win7 x64 Ultimate SP1
 
 

Lies, damned lies, and benchmarks

This says it all.
My System SpecsSystem Spec
23 Nov 2010   #10

 

"He made some minor changes to the test—changes that don't alter the result of the calculation the test performs and that, naively at least, should be treated as equivalent—and saw Internet Explorer 9 slow down considerably."

when you change the conditions, doesnt that change the results?

Im surprised that dev didnt run his changes against firefox, the article just says ie9's results "slow down considerably", what about other browsers?
My System SpecsSystem Spec
Reply

 Lies, damned lies, and benchmarks: is IE9 cheating at SunSpider?




Thread Tools



Similar help and support threads for2: Lies, damned lies, and benchmarks: is IE9 cheating at SunSpider?
Thread Forum
Is MS telling me LIES on anytime upgrade --weird symptom Installation & Setup
Low Memory warning: The computer lies? Performance & Maintenance
It lies, it says I'm not connected to wireless! Network & Sharing
Win7 says no ethernet cable is connected - it lies! Network & Sharing
Beneath this BSOD lies another BSOD Help and Support
Network Icon lies! Network & Sharing
Windows metrics source lies about identity News

Our Sites

Site Links

About Us

Find Us

Windows 7 Forums is an independent web site and has not been authorized, sponsored, or otherwise approved by Microsoft Corporation. "Windows 7" and related materials are trademarks of Microsoft Corp.

© Designer Media Ltd

All times are GMT -5. The time now is 11:32 AM.
Twitter Facebook Google+



Windows 7 Forums

Seven Forums Android App Seven Forums IOS App
  

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33