It's unfortunate that Microsoft (MSE) is @ (or near) the bottom. Seems like Microsoft should have a better footprint in security detection/protection. The free versions of AVG & Avast, both of which I have used in the past without problem, are a much better choice.
thank you
How do those conducting these AV comparison tests determine what is and what is not a virus? How would all companies that make antivirus apps agree that the same set of files are malicious? AV-Comparatives uses 12000+ files for its accuracy check. Perhaps those working on MSE don't think that all of 12000+ files are bad. Or there is some other reason for MSE's seemingly poor performance.
It would be nice/better if some researcher would take the time to publish a paper detailing the malicious actions of some files and then state which AV apps did not prevent those actions.
I did read your reply of three weeks ago but not well enough. You make a good point re: "how do those conducting these AV comparison tests determine what is and what is not a virus?" Especially with 12,000+ files.
I'm not MSE's biggest fan - as this post shows. MSE should never let an app take over the shell without warning the user.
But I wonder if companies like AVG/AVAST/BitDefender... study the AV-Comparitives testing method and write code that works well for those tests. This would mean that AV-Comparatives has a huge influence on what gets considered as malware. This might be a very good thing - IF those at AV-Comparitives are steering AV companies in the right direction (e.g. driving them to detect truly malicious files). Or, it could be a very bad thing (e.g. driving them to detect files that are not worthy of flagging*).
*assumes that the more files an AV company has signatures for, the slower things get.
Last edited by UsernameIssues; 30 Apr 2014 at 23:28. Reason: typo
MSE scored poorly in comparison to the other AVs whether they studied the "testing methods" or not.
MSE took the same tests everyone else took, and the test results speak for themselves.
Even if the companies had access to the tests and methods ahead of time like you suggest and MSE STILL scored last, what does that say for MSE? What's MSE's excuse, "The dog ate my homework"?
Not only that, but it's hardly fair to question the integrity of AV Comparatives or their testing methods without concrete evidence to show good cause.
They have a good reputation, and I've never seen or heard of any good reason to question the integrity of them or the service they provide.
Last edited by Dallas 7; 30 Apr 2014 at 23:55.
I did not mean to imply that any of these organizations making comparisons allowed any company to cheat. I'm not questioning anyone's integrity.
Let me give a concrete example of studying the testing methods:
MS knows that it does not flag the installer for Conduit as being bad.
Conduit is a class of malware that several AV apps ignore.
MS knows who scored high in a round of testing.
If those that score high flag file like Conduit, then MSE could too.
MS could opt to add tons of signatures to MSE.
"The test results still speak for themselves."
Then would you please answer my questions:
What constitutes a virus?
Was each of the 12000+ files used in the test a virus according you your definition?
Who decides what constitutes a virus?
Microsoft has said it bases it's detections on threats actually seen by it's users. How they can determine no new or unseen threats will be added. Supposedly they use user statistics, but how many agree to send such? A Guy
UsernameIssues,
First, some people get paid to define/detect viruses, and test AV programs, I don't.
I'm a consumer, not a designer. I have followed the AV test results of AV Comparatives for a long time, and doing so has served me very well. I have never found cause to question the validity or integrity of their services.
Second, if you're insinuating that AV Comparatives is playing favorites or loading the tests with files that don't even qualify as viruses, then you are indeed questioning AV Comparative's integrity for the sole purpose of trying to resurrect MSE from the bottom of the ash pile.
Third, I answered your question, it's your turn to answer mine:
MSE took the same tests everyone else took.
Even if the companies had access to the tests and methods ahead of time,
so did MSE, but they STILL scored last!
What's MSE's excuse, "The dog ate my homework"?
Last edited by Dallas 7; 01 May 2014 at 01:33.
Your answer to my first question is that you cannot answer it. You really don't have to answer it to see what I'm driving at. There are people at MS that get paid to decide what a virus is. There are people at AVG that get paid to decide what a virus is. There are people at AV-Comparatives that get paid to decide what a virus is. These people disagree about what a virus is and they always will disagree. There is no clear cut answer on what constitutes a virus and there never will be one that all parties agree to.
I am not questioning the integrity of those at AV-Comparatives. They include files they they define as viruses. They don't play favorites. However, they do not set the definition of what a virus is for the entire world.
I've followed AV test results since the early '90s when I had to pick/buy an AV app for our office. Also, malware is a part of my computing hobby. You saw how many hours I put in play with and studying the ransomware in my rant about how poor MSE's heuristics are. You should know by now that I'm not defending MSE - I'm merely attempting to get people to look at these AV comparisons in a different way.
There are classes of malware. MSE (and other companies) simply opt to not call certain classes as being bad. Doing that will always put them near the bottom of AV-Comparatives' testing. MS could decide overnight to flag everything that MS knows about.. but MS/MSE has made a business decision not to bloat their definitions with the files some other AV apps flag. This includes the installer for Conduit and files that are not in the wild. I prefer an AV app that flags Conduit's installer, so I'm looking for something other that MSE to install on the dozens of computers that I support for free.
BTW, Malewarebytes does the same thing. They remove the signatures for malware that is no longer in the wild. And while MBAM is not an AV app, the principle is the same. MSE could easily flag everything that MBAM flags and then some (e.g old malware), but MS/MSE opts not to - so as not to slow the computer down and to avoid false positives.
Just to make sure that I've answered your question:
What's MSE's excuse, "The dog ate my homework"?
They don't have nor need an excuse for coming in last or near the bottom. They opted to be there for good reasons. I just don't like or agree with all of their reasons.
Quote