Flawed Malwarebytes security update wipes out thousands of computers

[Kaktussoft]

SECURITY FIRM Malwarebytes has wiped out thousands of computers around the world with a faulty security update, mistaking legitimate system files as malware code.
The security firm confessed to the mistake in a blog post on Tuesday, and assured firms that the update has since been pulled.
"It saddens me to report that at around 3PM PST yesterday, Malwarebytes released a definitions update that disabled thousands of computers worldwide," wrote Malwarebytes Marcin Kleczynski.
"Within eight minutes, the update was pulled from our servers. Immediately thereafter, users flocked to our support helpdesk and forums to ask us for a fix."
The update definition made it so Malwarebytes protection software treated essential Windows .dll and .exe files as malware, stopping them from running and thus knocking IT systems and PCs offline.

Read more at:
Flawed Malwarebytes security update wipes out thousands of computers- The Inquirer

How to fix:
**Trojan.Downloader.ED** - Malwarebytes Forum

 

[lehnerus2000]

Testing before release?

The update definition made it so Malwarebytes protection software treated essential Windows .dll and .exe files as malware, stopping them from running and thus knocking IT systems and PCs offline.

Have these jokers even heard of the concept of "testing before release"?
Didn't they have any Windows PCs available?

 

[Kaktussoft]

The update definition made it so Malwarebytes protection software treated essential Windows .dll and .exe files as malware, stopping them from running and thus knocking IT systems and PCs offline.

Have these jokers even heard of the concept of "testing before release"?
Didn't they have any Windows PCs available?

Yes they have to test it on winxp, vista, win7, win8. Both x64 and x86 and ALL languages. Even with and without service packs. Update was online for only 8 minutes.... but it infected thousands of machines. Very strange

 

[Golden]

Posted yesterday : http://www.sevenforums.com/security...es-update-causes-massive-false-positives.html

 

[Layback Bear]

The good, bad and ugly.

The good was the problem was correctly quickly.

The bad is it still effect a bunch of computers and caused all kinds of hell.

The ugly; How did this update ever get past testing and inspection?
------
I do use Malwarebytes Anti Malware Professionas and missed the bad update. I update several times a day and somehow still missed the mess.

 

[stormy13]

Not to defend what happened but they aren't the first,

Avira Antivirus update cripples millions of Windows PCs | ZDNet

McAfee Anti-Virus Update Breaks Thousands Of PCs - HotHardware

and won't be the last security program to do such a thing.

 

[Layback Bear]

You are correct stormy13 it has happened before and I venture to say it will happen in the future.
For what ever reason I hold Malwarebytes Anti Malware to a high standard.

 

[Gary]

Can anyone spell Lawsuit?

 

[DavidE]

I understand mistakes like this can and do happen.
This is not the first "recent" FP with MBAM definition updates.
I got hit by the FP's in February, but I've read there have been 4 FP's since December 2012.
Why was nothing done to help prevent this problem before the latest FP.
This seems to be the worst MBAM FP issue, but the previous FP problems should have alerted them to take some action to prevent this.
I've lost a lot of confidence in using MBAM real-time protection, and I have it on all 5 of my personal systems.
:shock:

 

[Layback Bear]

I got hit by the first screw up and thought they would being paying more attention and not screw up again.

Maybe they got a IObit mole.

 

[Dallas 7]

Can anyone spell Lawsuit?

E-x-p-e-n-s-i-v-e

 

[tom982]

Have these jokers even heard of the concept of "testing before release"?
Didn't they have any Windows PCs available?

Before you go slating them, do you even know who these 'jokers' are? Some of the biggest names in the security community are behind Malwarebytes: Atribune (creator of VundoFix), Merijn (creator of HijackThis), Metallica (author of so, so many removal guides), S!Ri (creator of SmitfraudFix), screen317 (creator of Security Check), sUBs (creator of Combofix), Swandog46 (creator of The Avenger) just to name a few, of which there are many others. They're given countless hours of their time to the community and have asked for nothing in return; maybe you should think twice before criticising them for a small mistake in a free tool, unless you could do better? :huh:

 

[lehnerus2000]

Before you go slating them, do you even know who these 'jokers' are? Some of the biggest names in the security community are behind Malwarebytes: Atribune (creator of VundoFix), Merijn (creator of HijackThis), Metallica (author of so, so many removal guides), S!Ri (creator of SmitfraudFix), screen317 (creator of Security Check), sUBs (creator of Combofix), Swandog46 (creator of The Avenger) just to name a few, of which there are many others. They're given countless hours of their time to the community and have asked for nothing in return; maybe you should think twice before criticising them for a small mistake in a free tool, unless you could do better? :huh:

Tell that to the people whose PCs stopped working.
It's lucky it apparently didn't affect any mission critical software (e.g. hospitals, traffic control, etc.).

They couldn't load the update onto some Windows PCs in their lab and see if it worked?
I'm pretty sure I could have managed that much.

They don't have a list of "... essential Windows .dll and .exe files ..."?

Since these guys are security experts, they should be aware of the blunders committed by other AV companies (e.g. AVG, Avira, McAfee, etc.).

If anything, the fact that these guys are experts, makes it worse.

 

[Element7]

Good thing I didn't update!

 

[Lady Fitzgerald]

...They couldn't load the update onto some Windows PCs in their lab and see if it worked?
I'm pretty sure I could have managed that much...

Oh? You really think you can try to duplicate all the possible combinations of hardware and software running out there better than they can?

I'm sure they did test it in their labs before releasing it but there are so many combinations of software and hardware possible and actually running, there is no way they can be expected to anticipate every possible scenario. I'm amazed that they, M$, etc. do as well as they do.

 

[tom982]

The fact that they are experts in their field doesn't change matters, they're still human and these mistakes happen. What's more important is that they've learnt from this and have enforced various safeguards to prevent this from happening again:

Improvements to our Updating Process - Malwarebytes Forum

I can understand the frustration of the corporate users having hundreds of computers taken offline but for one home user of the software to complain about a small hitch in the excellent software that they provide for free, I think it's incredibly naive.

 

[lehnerus2000]

"... essential Windows .dll and .exe files ..."

...They couldn't load the update onto some Windows PCs in their lab and see if it worked?
I'm pretty sure I could have managed that much...

Oh? You really think you can try to duplicate all the possible combinations of hardware and software running out there better than they can?

I'm not an international company that sells software (I can't afford more than one PC).
Also, I didn't say that I could have tested it on every computer in the world (and I'm not blaming them for not testing it on every computer in the world).

Last time I checked AV programs were software based.
What does the hardware have to do with "... essential Windows .dll and .exe files ..."?

Are you suggesting that this update would have crashed Macs or Linux machines?

I'm sure they did test it in their labs before releasing it but there are so many combinations of software and hardware possible and actually running, there is no way they can be expected to anticipate every possible scenario. I'm amazed that they, M$, etc. do as well as they do.

They can't afford to have 100 VMs that they run from a group of servers?

Would you be so generous, if you bought a car and it crashed, because the factory forgot to fit wheel nuts?
"So your car crashed because we forgot the wheel nuts. The other 999,999 cars we made last year had wheel nuts. What are you complaining about? That's a good ratio in anyone's books."

The fact that they are experts in their field doesn't change matters, they're still human and these mistakes happen. What's more important is that they've learnt from this and have enforced various safeguards to prevent this from happening again: ...

Wrong.
The whole point of procedures is to prevent human error.
Why do you think pilots go through checklists before they take off?

This isn't the very first time this problem has occurred with AV updates.
They should have had procedures to prevent this happening (i.e. learnt form the mistakes of the other companies).

I can understand the frustration of the corporate users having hundreds of computers taken offline but for one home user of the software to complain about a small hitch in the excellent software that they provide for free, I think it's incredibly naive.

It didn't affect me.
I actually updated and ran MalwareBytes on my PC that night.

Maybe you don't get "slated" when you screw up, but most people do.

I hope you'll be that understanding when your surgeon goes "Oops", during your next operation.
After all he's only human.

 

[Phone Man]

1. We've installed a false positive shim server. This server will have virtual machines running a wide range of different configurations and operating system versions, to mirror the range of setups our customers run. Before an update gets pushed out, it will be tested on this server, on every configuration. If a false positive is detected, it will prevent our research team from uploading a database update.

2. We've modified the tools that compress and encrypt our definition updates. The false positives on Monday were not traditional, they were caused by a corrupted file that our encryption tool did not flag. We've made immediate changes to the tool and are testing it with a roll-out date to the entire research team by the end of the week.

Sounds like a good plan.

Jim

 

[lehnerus2000]

Agreed

1. We've installed a false positive shim server. This server will have virtual machines running a wide range of different configurations and operating system versions, to mirror the range of setups our customers run. Before an update gets pushed out, it will be tested on this server, on every configuration. If a false positive is detected, it will prevent our research team from uploading a database update.

2. We've modified the tools that compress and encrypt our definition updates. The false positives on Monday were not traditional, they were caused by a corrupted file that our encryption tool did not flag. We've made immediate changes to the tool and are testing it with a roll-out date to the entire research team by the end of the week.

Sounds like a good plan.

Jim

Agreed.

 

[badbreezer]

I agree and disagree with the comments in this thread. I guess like anything if yur a victim the shoe is reversed. I also have used MB (pro) for the longest and just today it actually removed some ROUGE files that were causing me grief ! With that being said i have the program set to run once weekly on a Wednsday which so far has worked more than satisfactory. More so now since i was able to avoid Mondays smash Still and all i dont think its reason to argue, let's keep it simple and wish those Luck that have to recover there PC's ! Overall MB is a great Proggie