During July 2009, a company called NSS Labs performed two separate browser security tests, which Amy Barzdukas, General Manager of Internet Explorer, told Ars that Microsoft had sponsored. Right off the bat, your suspicions have probably been raised, and rightly so. Internet Explorer 8 performed very well in all the tests and, while Microsoft insists that it had no impact on the results, we must still be cautious when examining the reports.
Before we go to the results, it’s worth noting that NSS Labs chose to test what it thinks are the most important types of security threats:
The most common and impactful ’security threats’ facing users today are socially engineered malware and phishing attacks. As such, they have been the primary focus of our initial research. While drive-by downloads and click-jacking are also effective attacks and have achieved notable publicity, they represent a smaller percentage of today’s threats.
According to Microsoft, the malware report is more important than the phishing report, so we’ve put it first. “We block 20 times more malware per day than phishing sites in IE8,” Barzdukas told Ars. IE8 block malware for approximately 1 out of 40 users every week, and approximately 1 of every 200 downloads is blocked as malicious.
Socially Engineered Malware Protection Comparative Test Results 2nd Edition
This test was actually NSS Labs’ second iteration to verify web browser protection against socially engineered malware. The first one was conducted in Q109 and published in March 2009. NSS Labs describes a socially engineered malware URL as “a web page link that directly leads to a ‘download’ that delivers a malicious payload whose content type would lead to execution.” Here’s how NSS Labs described its methodology for this test:
It is based upon empirically validated evidence gathered during 12 days of 24×7 testing, performed every 4 hours, over 69 discrete test runs, each one adding fresh new malware URLs. Each product was updated to the most current version available at the time testing began, and allowed access to the live Internet.
Here are the summarized results of this report:
The first graph is fairly self-explanatory: it shows the percentage of malware URLs that each browser was able to successfully detect and block. NSS Labs started with 2,171 potentially malicious URLs and then cut that down to 608 URLs as the rest “did not pass our validation criteria, including those tainted by exploits or that contained invalid samples.”
The second graph shows the response time of how long it took browsers to block a threat once it was introduced into the test cycle. Generally, at least half of a browser’s total protection was achieved in the zero hour. Internet Explorer 8 continued to add as much as 41 percent of additional protection over the course of the test while other browsers added between just 0 percent and 16 percent.
Phishing Protection Comparative Test Results
This test was NSS Labs’ first comprehensive test of web browser phishing protection. NSS Labs describes a phishing URL as follows: “The URL both falsely impersonates another entity and attempts to trick the user into disclosing personal information via a Web form.” Here’s how NSS Labs described its methodology for this test:
This report is based upon empirically validated evidence gathered by NSS Labs during 14 days of 24×7 testing, performed every 4 hours, over 80 discrete test runs, each one adding fresh new phishing URLs. Each product was updated to the most current version available at the time testing began, and allowed access to the live Internet.
Here are the summarized results of this report:
The first graph is fairly self-explanatory: it shows the percentage of phishing URLs that each browser was able to successfully detect and block. NSS Labs started with 856 potentially phishing URLs and then cut that down to 593 URLs as the rest “did not pass our validation criteria, including those tainted by exploits (not part of this test).”
The second graph shows how long it took the browsers under test to block the threat once it was introduced into the test cycle. NSS Labs notes that although Firefox, Safari, and Chrome share the same SafeBrowsing API, their results are very different, which speaks to the impact of different implementations.
Conclusion and Microsoft commentary
When Internet Explorer 8 was released in March 2009, we noted that Microsoft had made significant progress, including in the area of security. Which is good, since the company maintains the dominant position in terms of browser market share, and as a result it will be the biggest target for various Internet threats.
When we talked about IE6 and Microsoft’s stance on not dropping support anytime soon, Barzdukas told us that “our biggest competition, in many ways, is our own past version” and that it was important to “help consumers understand the value of moving to modern browsers.” Touting how well the browser does in the area of security is apparently one of the ways Microsoft plans to do this. Another is simply waiting for Vista’s successor. “We think that with Windows 7, we will see much more traction with those IE6 users moving on,” Barzdukas noted.
Microsoft has tried before to market IE8 as a superior browser to its predecessors and to the competition. In June 2008, the company launched a “Windows Internet Explorer 8: Get the facts” website that received much criticism because it was obviously biased. In March 2009, the company released its own tests that showed IE8 loading many websites faster than two open source browsers, Firefox and Chrome. With these NSS Labs reports, the company is trying to show that third-parties also find IE8 to be a solid competitor, but the fact that the software giant sponsored the reports blunts their impact (the way that one designs the test has a huge impact on the results).
In terms of the future, Barzdukas told Ars what we already know: that Microsoft claims to be taking HTML 5 very seriously. The specification is being authored collaboratively by representatives of numerous companies and organizations, including all of the leading browser makers. When we steered the discussion to IE9, though, Barzdukas told Ars that Microsoft is still “in the early stages, taking a lot of feedback, and we are hammering out the plan.”
Update
Rick Moy, president of NSS Labs, sent us a follow-up e-mail to tell us that it was Microsoft’s online security engineering team (not marketing) that hired NSS Labs to do recurring benchmark testing so they could improve their services. Only once Microsoft’s security engineering team saw the results did it send the details over to the marketing department.
In terms of sponsorship of the reports, “this stuff is expensive to do right, and we need to monetize it somehow,” Moy told Ars. “We invited Google, Mozilla, Apple, Opera to participate, but they didn’t even bother to respond, except for Opera, which stated they “don’t really focus on malware.”
about author ..
Please look for a later launch.