Solved 45-50 Machines BSOD Intermittently 0x0000000a

[technaustin]

I have been through many steps, Ill try to list them all though please forgive me if I forgot a few. This is a windows 7 domain environment, with us being the IT support. 45/50 of these machines BSOD with a 0x0000000a error. Some of the machines do not BSOD but freeze/pixelate. The company purchased these PC's before they hired us. The errors started (Supposedly) by a warning of event 51 in the event viewer which is a paging file error on the main Drive (drive0). This error is on almost every machine. HP Insists it is not hardware, now it is too late to matter. The machine is an HP 405 G1. The machines have 4gb of ram, but one of the techs upgraded them all to 8 hoping to solve the issue. It hasn't. I have ran driver verifier, hardware diagnostics, Windows Updates, bluescreen view. The only commonality i believe is that they all have the same BSOD when they crash (except sometimes they don't BSOD they pixelate with NO minidump left behind). This is rather annoying to the company and also to us. Lets just say i prefer intel to say the least. There has been a recent Bios update 80.07 that supposedly fixes an erratum. I don't think it has solved anything. We have updated graphics drivers, chip set drivers. Removed AMD Sata controller to try MS AHCI. Months of BSOD's that only have 0x000000a in common. My thought is this machine doesn't have great performance, and it is possible it can't handle business type work with a few proprietary software, but regardless a machine shouldn't crash because its slow. The CPU maxes out 70-100% with as little as a flash video, in fact the dm log maxed it out. So there are 3 problems, which we think all are in conjunction Event 51 Error, BSOD 0x000000a, and sometimes Pixelated freeze. The machine which I have sent the files for is now not giving us BSOD's as much, but starting to pixelate when it crashes( so its about 1/2 of each now). I'm hoping a Windows expert can see what I cannot. I typically don't like the shotgun method of troubleshooting but these machines haven't given us much direction. We have tried HP image non-hp image etc. You get the picture. Any other steps I could try would be much appreciated, this is my first post ever for PC related things so I hope there is enough detail with the files and notes. Thanks for your help. (There is probably a lot more that i have done over the last month but these are the main things). The last option i have "heard" is to increase in regedit the time for graphics card timeouts, or lowering the clock speed of the GPU, which in my opinion is crazy to have to do to new machines that are less than a year old (about 9 months) or any machine for that matter that is sold as a business machine. I can't believe HP calls this a business machine, my phone is faster. Anyways.. Have no idea if this GPU lowering is a resolution, but that's the only thing I can think of that I haven't tried. Here is the steps and notes the company had tried before we arrived:

((In July we had purchased 50 HP ProDesk 405 G1 MT. 45 out of the 50 are exhibiting the follow error messages in the system event log at some point:

"An error was detected on device \Device\Harddisk0\DR0 during a paging operation."
(See screen shot below)

Roughly 20% of the pc experience this issue 2-3 times every 30 days- (including mine!). The rest of the PC's are less frequent - once - to every other month this issue will occur.

I have opened several cases w/ HP and we have done the following steps to see if this resolves the issue:

Hardware replacements:

-
Sata Cables
-
Hard Drive
-
Memory
-
Power Supply
-
Motherboard
-
Graphic card

OS / software modifications:

-
Factory restore HP image - clean
-
Ran HP hardware diagnostics - clean
-
Ran extensive 12+ hardware test - clean
-
BIOS update
-
Disabled Prefetch - Changed from 3 to 0 HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory
-
Ran SFC scan on hard drive
-
Turned Off Sleep Mode Display
-
2/18 Error msg captured in Procmon. Maybe audio/graphic driver
-
Reinstalled audio driver
-
Reinstalled graphic drive
-
Disabled Sound and stop Realtek Audio service and disabled
-
Uninstalled AMD Catalyst software suite. ( We ended up doing this on all PC's - because software was affecting performance)
-
Unistalled Vipre AV - and installed AVAST on one PC as part of our troubleshooting
-
Disabled write-caching on hard drive

** I did read in one forum that someone had contacted in Microsoft and they had a patch for this particular. I have not done that yet. ))

Specs:
AMD E1-2500 1.4ghz
4gb (most upgraded to 8gb)
500gb 75% Seagate HDD (25% WD)
Windows 7 Pro x64
Radeon APU (8200m)


Any help would be greatly appreciated.

 

[Arc]

BugCheck A, {1e9ffa38, f, 0, fffff80002c21b60}

*** WARNING: Unable to verify timestamp for cpuz137_x64.sys
*** ERROR: Module load completed but symbols could not be loaded for cpuz137_x64.sys
Probably caused by : cpuz137_x64.sys ( cpuz137_x64+2073 )

Followup: MachineOwner
---------

The BSOD was caused by CPUz driver. It is basically a part of Cupid CPUz, and also used by Pirifirm Speccy. But neither of them two is noticed by me in your system.

It may be that I am missing. If you have either of them, uninstall them.

If you dont have any of these two installed, there may be three possibilities.

First: It may be that the driver is used by any other hardware monitoring program/gadget. Please uninstall if you have any of such stuff installed.

Second: It may be that the driver is operating from temp (many a time it is noticed). Download, Install and Run CCleaner .... it will take care of the temp files properly.

Third: As the image path is not showing the regular path, it might be a virus as well. Scan the system for possible virus infection with the following programs.

• Anti-rootkit utility TDSSKiller
• Windows Defender Offline

Let us know what you found.

 

[technaustin]

That's funny. Thank you very much for the quick reply. We are pretty sure its not malware, we have ran scans on the machines (and there is 50), but i will check again. This machine I have run different things to test with, but i don't recall using CPU z. I have used Autoruns and such to look at drivers as well as some other programs. I will look on the other machines for monitoring software's etc. Each machine (the other 49) already have CCleaner, I removed it from this one thinking it may be causing a problem. Ill Run it again see if it helps. Ill get started by removing any extra software, and run the two softwares for rootkits. (the only possibility for malware is maybe a infected image ? ) Thanks again. I will do that now.

 

[technaustin]

HWMonitor is installed on this machine that is from the same company. I installed it to monitor temps, but the problem was happening before this. I will remove it just in case. Right now I cannot run defenfer offline, as I am not near the machine for a few days, but TDDSkiller found 0 threats. Remember this is happening to about 40 or so other machines 1-2 times per week. I will see if one of the local IT people can run defender offline to double check though. So now i will remove the extra softwares and run CCleaner. After should i re-send the dump files?

 

[technaustin]

Removed Burn-In test 7.1, deleted HWMonitor, and Spybot SD (which i used earlier to check for malware). Installed CCleaner ran it to clean registry / temp files. (earlier Spybot also found nothing) I was told this was a factory image from HP, then they installed their company software's (the rest of the PC's do not have HP image but a custom image but still have this issue). What do you think about Dameware, it is the internal IT software for remote access, i just ask because I noticed some of the drivers are very old? Thanks again. I will update if anything else happens.

 

[technaustin]

BSOD on next machine over

This BSOD happened today, about 20 minutes ago, but no mini dump was left, this is NOT the PC uploaded, (56) same make model image etc, just different user, and an example of what is happening to all of these units except a couple. I wish this unit would leave a dump as it has been crashing a lot lately. Not sure if you want this information. The internal technician sent me a photo of it. It is always 0x0000000a, and usually doesn't leave a dump file. I'm perplexed. He is going to run the Defender Toolkit today on the machine that was uploaded. I will update with anything else. Each one of these machines is getting Event 51 error a warning about the paging file to memory i believe, does this have any relation to whats happening? I see event warning 51 on PC's a lot, but here its happening on the main drive not the secondary drives, which i have found to be more common.

 

[Arc]

HWMonitor or any other hardware monitoring program may include that problematic CPUz driver. I would suggest to uninstall them all.

About the outdated drivers, you may attempt to update them: http://www.sevenforums.com/tutorials/314096-drivers-find-update-outdated-drivers.html

And about the BSOD on the other machine, unfortunately nothing can be told precisely without seeing a crash dump. Carrona.org mentions the above causes of a stop 0xA BSOD:

STOP 0x0000000A: IRQL_NOT_LESS_OR_EQUAL

Usual causes: Kernel mode driver, System Service, BIOS, Windows, Virus scanner, Backup tool, compatibility

Add one more thing there, a failing RAM. So a lot of possible places to look at.

 

[technaustin]

Gotcha, thanks ill look into that. Yes we suspected bad ram but tested. I will do my best to get another minidump file, let you know if i find anything else. I've looked through driver view, but will follow this guide looks helpful. Thanks!

 

[technaustin]

3 more machines BSOD since yesterday, only one left a minidump having to do with USBhub.sys, so removed all USB drivers and restarted, then updated the USB drivers from HP's website. Could this be the issue causing all of these machines to crash?

 

[Arc]

Hi technaustin.

Do to things.

Upload the list of all the third party drivers Using NirSoft DriverView :

• Download and execute Driverview
• View > Hide Microsoft Drivers
• Edit > Select all
• File > Save Selected Items
• In the Save dialog, Set the path to desktop, Put "Driverview" in the name field, and save.
• Zip the .txt file and upload it.

Test your RAM modules for possible errors.

How to Test and Diagnose RAM Issues with Memtest86+
​

Run memtest86+ for at least 8 consecutive passes.

If it start showing errors/red lines, stop testing. A single error is enough to determine that something is going bad there.

Let us know the results.
________________________________________________________

*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

Use !analyze -v to get detailed debugging information.

BugCheck FE, {8, 6, 6, fffffa8005fb4640}

Probably caused by : usbhub.sys ( usbhub!UsbhWaitEventWithTimeoutEx+3aa )

Followup: MachineOwner
---------

 

[GokAy]

Hey there. Just an idea while you are trying to diagnose.

You have a couple PCs without an issue and all are same?
Maybe cloning from a working one with sysprep while keeping drivers would work?

You will have to decide on feasibility of such an act though. Seems a lot of work.

Take care

 

[technaustin]

Thanks, I think the internal IT will run the Memtest at end of day, if not I will head out there monday, and I will send you the drivers list. As much as i don't think its a memory issue, the one like this that pixel-ate make me think it might be on some of them, like this one. Here is the updated driver list.

 

[technaustin]

So I thought maybe that was an issue, they have a Sysprep image that they loaded from. But the machine that I started diagnosing, they gave me with factory HP image, to make sure. So we have tried both, that was one of our first thoughts, a corrupt image, that's a great idea, if we can find one that doesn't crash. Ill ask the guys. At this point the with the amount of BSODs, if it was for sure because of an image, we would re-do it.

 

[Arc]

Almost all the drivers are outdated there. QLogic, Adaptec ... all those.

If memtest86+ comes free of errors, teh best deal for you would be perform a http://www.sevenforums.com/tutorials/219487-clean-reinstall-factory-oem-windows-7-a.html

 

[technaustin]

You were correct from the start, my boss found an incompatibility between this model and the CPU_Z driver used in our monitoring software, thanks so much, as soon as he mentioned that I knew what to look for thanks to your expertise. BSOD issue solved, now gotta figure out why they crash /pixelate sometimes. There are still outdated drivers we are working on, but ever since we have removed the sensor feature we haven't had any issues with BSOD.

 

[Arc]

Good news.