Windows 7 Forums

Welcome to Windows 7 Forums. Our forum is dedicated to helping you find support and solutions for any problems regarding your Windows 7 PC be it Dell, HP, Acer, Asus or a custom build. We also provide an extensive Windows 7 tutorial section that covers a wide range of tips and tricks.


Windows 7: Lies, damned lies, and benchmarks: is IE9 cheating at SunSpider?

20 Nov 2010   #1
JMH

Win 7 Ultimate 64-bit. SP1.
 
 
Lies, damned lies, and benchmarks: is IE9 cheating at SunSpider?

Quote:

The SunSpider JavaScript performance benchmark, devised by the developers of the WebKit browser engine, is used and quoted widely as a measure of browser scripting performance. A surprising result was recently noticed by a Mozilla developer, Rob Sayre, looking at Internet Explorer 9's performance in this test. On one of the many subtests it performs, Internet Explorer 9 was finishing the subtest almost instantly.

In and of itself, that's not necessarily very interesting; several of the subtests in SunSpider are near-instant in the browser. However, it piqued the developer's curiosity. He made some minor changes to the test—changes that don't alter the result of the calculation the test performs and that, naively at least, should be treated as equivalent—and saw Internet Explorer 9 slow down considerably. He filed a bug against Internet Explorer.

Sayre's bug report was conservative—he suggested that an optimization that Internet Explorer 9's Chakra JavaScript engine was performing was fragile, and was easily disabled by minor alterations to the code that it should disregard. Coverage earlier today of the same issue was less guarded: Internet Explorer 9 was accused of cheating in the test. The allegation is that Microsoft has built a specific optimization into Chakra that detects, and bypasses, the specific code in SunSpider, but which has no other purpose. In other words, the optimization will not do anything to improve the browser's performance in any other scenario.
Lies, damned lies, and benchmarks: is IE9 cheating at SunSpider?


My System SpecsSystem Spec
.

21 Nov 2010   #2
windows7user

Windows 7 Professional x64
 
 

I've read about this a couple of days ago. Microsoft isn't really cheating. JavaScript engine in IE9 is just a bit more intelligent, in that it can detect useless code and skip executing it. It's called dead code optimization and the article explains it. In IE9's case, the JavaScript engine has been optimized to do very well in the SunSpider JavaScript benchmarks.

The real question here is that since it is clearly shown that a browser can be tweaked to do well in certain benchmarks, how reliable are benchmarks?
My System SpecsSystem Spec
21 Nov 2010   #3
Windows i7 920

Windows 8 Pro x64
 
 

I use real situations for testing, and such testing has shown Google Chrome to be the fastest for the sites I visit.
My System SpecsSystem Spec
.


22 Nov 2010   #4
Lee

Win 7 Pro x64, VM Win XP, Win7 Pro Sandbox, Kubuntu 11
 
 

Who really cares? This is just plain silly. Oh Teacher, Steve is cheating again. As if no one else does it. Oh darn, but Google doesn't, and I heard that Apple is always telling the truth.

Get over it, they all have their own way of testing; this helps them look good in the eyes of those who are not technically inclined. Beside once someone has chosen a application or browser it is what they have a tendency to stay with.
My System SpecsSystem Spec
22 Nov 2010   #5
Lemur

Systems 1 and 2: Windows 7 Enterprise x64, Win 8 Developer
 
 

Video cards did (and still do) optimize for benchmark tests. They're playing the same old song. It's not that big of a deal with browsers, as no expense is laid out. It's rougher with video cards, as you buy it, you own it.
My System SpecsSystem Spec
22 Nov 2010   #6
ken9122

Win7 x64 Ultimate SP1
 
 

Quote   Quote: Originally Posted by Lee View Post
Who really cares? This is just plain silly. Oh Teacher, Steve is cheating again. As if no one else does it. Oh darn, but Google doesn't, and I heard that Apple is always telling the truth.

Get over it, they all have their own way of testing; this helps them look good in the eyes of those who are not technically inclined. Beside once someone has chosen a application or browser it is what they have a tendency to stay with.
Exactly
My System SpecsSystem Spec
22 Nov 2010   #7
Lemur

Systems 1 and 2: Windows 7 Enterprise x64, Win 8 Developer
 
 

Quote   Quote: Originally Posted by ken9122 View Post
Quote   Quote: Originally Posted by Lee View Post
Who really cares? This is just plain silly. Oh Teacher, Steve is cheating again. As if no one else does it. Oh darn, but Google doesn't, and I heard that Apple is always telling the truth.

Get over it, they all have their own way of testing; this helps them look good in the eyes of those who are not technically inclined. Beside once someone has chosen a application or browser it is what they have a tendency to stay with.
Exactly
We do tend to fall in love with our browsers. And, speaking as one, I like to see it do well in the benchmarks. If it becomes painfully obvious it is lacking, I would consider switching. Different browsers, different search engines, different platforms. Ain't computing great!
My System SpecsSystem Spec
22 Nov 2010   #8
mickey megabyte

ultimate 64 sp1
 
 

Quote:
when what's important is not measurable, what's measurable becomes important.
they all lie.
My System SpecsSystem Spec
22 Nov 2010   #9
ken9122

Win7 x64 Ultimate SP1
 
 

Lies, damned lies, and benchmarks

This says it all.
My System SpecsSystem Spec
23 Nov 2010   #10
dmex

 

"He made some minor changes to the test—changes that don't alter the result of the calculation the test performs and that, naively at least, should be treated as equivalent—and saw Internet Explorer 9 slow down considerably."

when you change the conditions, doesnt that change the results?

Im surprised that dev didnt run his changes against firefox, the article just says ie9's results "slow down considerably", what about other browsers?
My System SpecsSystem Spec
Reply

 Lies, damned lies, and benchmarks: is IE9 cheating at SunSpider?




Thread Tools





Similar help and support threads
Thread Forum
Low Memory warning: The computer lies?
I have windows 7 64 home premium installed on the following computer: Intel i5 2500K (3.30Ghz, 4core, 6MB cache) Asus P4H67 8 GB Ram 2TB Hard drive and when running a specific application (that task manager says is only using about 500 MB), Windows gives me a Low Memory warning, even...
Performance & Maintenance
It lies, it says I'm not connected to wireless!
Hey everyone, having a bit of an odd situation. The wireless network icon is showing a red cross, normally meaning that there is an issue with a network that you're trying to connect to. However, I am connected perfectly and 100%! In the Network and Sharing Centre it shows issues and when I...
Network & Sharing
Win7 says no ethernet cable is connected - it lies!
Hello all - first post! My internet worked fine for months until about 3-4 weeks ago I stopped grabbing a connection. The link light on my modem (Linksys CM100) doesn't show a connection, though it says I'm online. Win7 says I don't have an ethernet cable connected and I've tried 3 different...
Network & Sharing
Beneath this BSOD lies another
Hi Folks, Have been getting random BSODs over the past few weeks, minidumps attached plus system info. Nothing seems to cause them in terms of my actions. The BSODs are all very similar according to BlueScreenView. Hibernation and sleep not enabled. So far: -Have cleaned the internals (cpu...
BSOD Help and Support
Network Icon lies!
I did some searching and found 1 other post where a user seemed to of had the same issue as me. The network icon shows the computer screen with the red X on it, but he was on the network. Restarting the system seemed to have fixed his issue, but I have restarted a dozen times, and it is stilll...
Network & Sharing
Windows metrics source lies about identity
Craig Barth, the chief technology officer of Florida-based Devil Mountain Software, a company that makes and markets Windows performance metrics software, is, I have discovered, nobody. He doesn't exist. Barth is, in fact, a nom de plume, which is a fancy, French way of saying "alias." The real...
News

Our Sites

Site Links

About Us

Find Us

Windows 7 Forums is an independent web site and has not been authorized, sponsored, or otherwise approved by Microsoft Corporation. "Windows 7" and related materials are trademarks of Microsoft Corp.

© Designer Media Ltd

All times are GMT -5. The time now is 23:09.

Twitter Facebook Google+



Windows 7 Forums

Seven Forums Android App Seven Forums IOS App