Solved Random Freezes, BSOD, error 0x116

[Baasha]

*bump!

please help..

Without verifier we would be just speculating. In what way can we help?

Okay, I will try verifier again.

However, lately, I cannot even log on to the desktop! The log on "sound" comes on but the screen/monitor just goes into sleep mode. When I reboot, it comes back on but when I put the password in, it takes a LONG time to get to desktop. The "Welcome" with the spinning circle just sits there for at least a minute and then the desktop comes on. Any idea what that could be? Is it most likely a software issue?

I will turn on Driver Verifier tonight and see what happens.

I might end up doing another fresh install of the OS but I'll try Verifier again. :huh:

This is so maddening.

 

[writhziden]

You can find out if it is a software issue by running a clean startup: http://www.sevenforums.com/tutorial...ation-conflicts-performing-clean-startup.html.

 

[Baasha]

You can find out if it is a software issue by running a clean startup: http://www.sevenforums.com/tutorial...ation-conflicts-performing-clean-startup.html.

Okay I did a Clean Boot and the system booted up fine. Although, I still have this issue where I put in the password and it takes a LONG time to get to the desktop! This happens even in Safe Mode so I know it's not some "service" or "startup program" that's causing it.

When I did a fresh install of the OS (and before I had any issues), once I put in the password, the desktop would appear almost instantaneously after the famous "beep".

The other thing I am having trouble with was that the audio devices were getting switched automatically. I have 2.1 speaker system connected by an Optical output from the motherboard to an A/V Receiver (S/PDIF output) and I also have my headphones plugged in to their respective jacks (green and pink) on the back of the motherboard. When I use the headphones, I always right-click the speaker icon in the Taskbar and select "Speakers" and press "Set as Default" and it would switch to those. When I wanted the speakers, I would click the "S/PDIF" and click "Set as Default" and it would switch back. Now, when I try to set the output to the S/PDIF, it keeps automatically switching to the "Speakers"; in other words, I don't get any sound from the 2.1 system; only the headphones.

I then went to the ASUS website and downloaded their "AUDIO" drivers for the motherboard and installed them. That installed some 3rd party program called SoundMAX. Now the S/PDIF works fine but I've NEVER had to install anything else to do this before. I always had the "High Definition Audio Device" in the Device Manager and just used that (since I'm using on-board sound and no sound card).

Besides, I also turned on Driver Verifier now and there are NO BSODs! I even played a game (Sniper Elite V2) for about an hour and it didn't even hiccup once! I played it at max settings so I think that pretty much rules out any driver conflict right?

So right now, I'm running 3-Way SLI where the 3rd, 5th, and 7th PCI-E slots (PCI-E 2.0 x16) are occupied. The moment I install the 4th card and populate the 1st PCI-E slot, the system causes all sorts of problems. However, I tested that slot by itself (with 1 card in the system) and there are no issues at all!

I've pretty much narrowed it down to this:

If I install GPUs in the 1st and 3rd slots (either SLI or 3-way SLI), the system crashes, freezes, and is not working properly.

If I use any of the 4 PCI-E 2.0 x16 slots (1, 3, 5, 7) individually, there are no issues. Even if I use slot 1 and 5 for SLI, there are no issues. If I use slot 3 and 5 for SLI, there are no issues. :sarc:

So, it seems like ONLY when I use BOTH slot 1 and 3 together, the problem(s) arise(s). What could this mean? If the slots individually work, why are they having issues when I populate them together? :shock:

 

[writhziden]

Could be an issue with the SLI, though if it works in the other slots, that is not too likely. More likely it is an issue with the motherboard itself. Contact ASUS and see what they recommend: ASUSTeK Computer Inc. -Support- Contact us.

 

[Baasha]

Could be an issue with the SLI, though if it works in the other slots, that is not too likely. More likely it is an issue with the motherboard itself. Contact ASUS and see what they recommend: ASUSTeK Computer Inc. -Support- Contact us.

I will contact ASUS on Monday since it's too late to do so today.

After running Driver Verifier for the whole day, I did get one BSOD; it was BCC 0x50.

I've attached the dump file so can you please let me know what it says? Is there a particular driver that's causing this issue?

The other thing that happened this evening that was weird is that when I plugged in my other monitors (I usually run 3 in Surround mode but was testing the system with just one), the desktop came on but the taskbar automatically was moved to the top(?)!! How/why did this happen? I had to "unlock" the taskbar and move it back to its original position. This happened three times so far.

Also, can you tell me what I can do about the long load times (from password screen to desktop)?

 

[writhziden]

Your Verifier enabled crash indicates Kaspersky is running at the time of the crash. Recommend removing Kaspersky with the instructions in Removal tool for Kaspersky Lab products and installing Microsoft Security Essentials - Free Antivirus for Windows to test. It is usually a false positive when Kaspersky is blamed, but it does hinder us being able to find the true cause when it is installed. You can probably install it again once we have narrowed down the true cause.

0x50 usually indicates a memory problem; often it is related to a driver or program accessing memory it is not supposed to, but it could be the RAM itself, so you may want to check the RAM again. You never answered zigzag3143's question about whether you have tested your RAM one by one. What I would suggest is:

• Disable Verifier as described by zigzag3143
  
  
• Run with one RAM module in one slot. See if you get crashes.
  
  
• If you crash, try another slot and keep trying slots as you get crashes.
  
  
• If all slots fail, set that module aside as suspected bad and try another module.
  
  
• Once you find a module and slot that work, test that module in all slots to find any bad slots, and test all modules in good slots to find any bad modules.
  
  
• If none of the slots and none of the modules provide stability, the problem is likely another piece of hardware or a driver.

To test the RAM, do your normal routine in Windows for twice as long as it usually takes to get a crash, or do the things you normally do that cause crashes.


Also, did you run Prime95, FurMark, and the Video memory test from the http://www.sevenforums.com/crash-lo...op-0x116-video_tdr_error-troubleshooting.html instructions? What were the results? What temperatures are you seeing at idle and what temperatures are you seeing under stress?

We need more information about the diagnostics done and the results than have been provided.

 

[Baasha]

Your Verifier enabled crash indicates Kaspersky is running at the time of the crash. Recommend removing Kaspersky with the instructions in Removal tool for Kaspersky Lab products and installing Microsoft Security Essentials - Free Antivirus for Windows to test. It is usually a false positive when Kaspersky is blamed, but it does hinder us being able to find the true cause when it is installed. You can probably install it again once we have narrowed down the true cause.

I removed Kaspersky as it never started after a reboot; I kept getting an error from "avp.exe" and it referenced some "memory" that could not be read. The program never launched so I just uninstalled it completely.

With MSE installed before I tried Kaspersky, I got HUNDREDS of errors in Event Viewer from MSE ("...OOBE stopped working 0xC00000D"). I don't know how or why this was happening. I was never able to fix that even after I read about a suggestion that I should delete "oobe.etl" file to reset it. So I went ahead and uninstalled MSE and tried Kaspersky but obviously, that was a disaster as well.

0x50 usually indicates a memory problem; often it is related to a driver or program accessing memory it is not supposed to, but it could be the RAM itself, so you may want to check the RAM again. You never answered zigzag3143's question about whether you have tested your RAM one by one. What I would suggest is:

• Disable Verifier as described by zigzag3143
• Run with one RAM module in one slot. See if you get crashes.
• If you crash, try another slot and keep trying slots as you get crashes.
• If all slots fail, set that module aside as suspected bad and try another module.
• Once you find a module and slot that work, test that module in all slots to find any bad slots, and test all modules in good slots to find any bad modules.
• If none of the slots and none of the modules provide stability, the problem is likely another piece of hardware or a driver.

To test the RAM, do your normal routine in Windows for twice as long as it usually takes to get a crash, or do the things you normally do that cause crashes.

I did indeed test the RAM one stick at a time. I first ran them together for 7 passes and they had no errors. None of the sticks are bad; I tried each of the six slots as well.

Yesterday, I was running 3-Way SLI with PCI-E slots 3, 5, and 7 were occupied. I had Driver Verifier running and played several games, watched videos etc. There were NO problems other than the really long boot times (checked Event Viewer with Custom View for "Boot Times" and it had values of 93 - 96 seconds). The only time I got a BSOD (twice, both BCC 0x50) was when Kaspersky was trying to update its database.

Since the system worked fine with 3 cards and there were no driver conflicts, I decided to reimage the system to try and start from scratch with FOUR cards.

I installed the fourth GPU and re-imaged the computer. Then, after the initial reboot, it went to POST and then the monitor just went to sleep; there was no video! I tried several hard-reboots and restarts but the computer just refuses to reboot or start Windows. On a few of the reboot attempts, I kept getting a message that read, "Overclocking Failed! Press F1 to enter Setup...". So I went into BIOS and reset everything to default settings and the computer still refuses to boot into Windows! Since this is right after a Windows re-image, it is most likely a hardware issue right?

Also, did you run Prime95, FurMark, and the Video memory test from the http://www.sevenforums.com/crash-lo...op-0x116-video_tdr_error-troubleshooting.html instructions? What were the results? What temperatures are you seeing at idle and what temperatures are you seeing under stress?

We need more information about the diagnostics done and the results than have been provided.

I ran Prime95, Intel Burn Test, and OCCT and they all worked fine and passed without errors even at my OC settings; 4.41GHz @ 1.30 V-Core. I pretty much ruled out the CPU being faulty after that.

My CPU temps at idle (OC settings) are in the high 20s Celsius; 27 - 30C. Under 100% load, the CPU goes to 72C on the hottest core.

The GPUs all maintain temps of mid-30 C idle and 65 - 75C across the four GPUs under 99% load.

Many of the BSODs before were indeed 0x116 which is definitely GPU/video driver related. After testing each GPU individually and each PCI-E slot individually and having NO errors/problems, the problem only arises when PCI-E slots 1 AND 3 are used at the same time.

What's really frustrating is that even after a system re-image, I can't even boot into Windows!

 

[zigzag3143]

Avp.exe is Kaspersky so if you are still getting errors from it after you removed Kaspersky, you didnt remove it completely.

 

[Baasha]

Avp.exe is Kaspersky so if you are still getting errors from it after you removed Kaspersky, you didnt remove it completely.

I got the "avp.exe" errors when Kaspersky was still installed. I removed it completely and there were no errors afterwards.

I re-imaged the system but can't boot to Windows now!

 

[writhziden]

Try resetting the hardware/software connections with the BIOS:

1. Shut down and turn off the computer.
2. Unplug the computer from the wall or surge protector (then remove the battery if it is a laptop).
3. "Remove the computer from any port replicator or docking station, disconnect
   cables to printers or devices such as external monitors, USB memory sticks or SD cards, headset or external speakers, mouse or auxiliary keyboard, turn off WIFI and Bluetooth wireless devices." (Use Hard Reset to Resolve Hardware and Software Issues HP Pavilion dv5000 Notebook PC series - HP Customer Care (United States - English))
4. Hold down the power button for 30 seconds. This closes the circuit and ensures all
   power from components is drained to clear the software connections between the BIOS
   and hardware and clear any corruption in the temporary memory.
5. (If it is a laptop, plug the battery back into the laptop and then) Plug the computer back into the wall. Do not reconnect any unnecessary peripherals; monitor, keyboard,
   and mouse should suffice and be the only peripherals reconnected.
6. Turn it on to reinitialize the software connections between the BIOS and hardware

You could also try clearing the CMOS by loading setup defaults or optimized defaults in the BIOS exit screen.


Post back after doing each of these steps whether you are then able to boot into Windows.

 

[Baasha]

Try resetting the hardware/software connections with the BIOS:

1. Shut down and turn off the computer.
2. Unplug the computer from the wall or surge protector (then remove the battery if it is a laptop).
3. "Remove the computer from any port replicator or docking station, disconnect
   cables to printers or devices such as external monitors, USB memory sticks or SD cards, headset or external speakers, mouse or auxiliary keyboard, turn off WIFI and Bluetooth wireless devices." (Use Hard Reset to Resolve Hardware and Software Issues HP Pavilion dv5000 Notebook PC series - HP Customer Care (United States - English))
4. Hold down the power button for 30 seconds. This closes the circuit and ensures all
   power from components is drained to clear the software connections between the BIOS
   and hardware and clear any corruption in the temporary memory.
5. (If it is a laptop, plug the battery back into the laptop and then) Plug the computer back into the wall. Do not reconnect any unnecessary peripherals; monitor, keyboard,
   and mouse should suffice and be the only peripherals reconnected.
6. Turn it on to reinitialize the software connections between the BIOS and hardware

You could also try clearing the CMOS by loading setup defaults or optimized defaults in the BIOS exit screen.


Post back after doing each of these steps whether you are then able to boot into Windows.

Okay, I removed the 4th GPU and started the computer and as expected, it booted right up to Windows!

There is something definitely off when I install the 4th GPU; or actually, when I insert GPUs into the 1st and 3rd PCI-E slot. Could it be a power issue? I'm using 2 EVGA Power Boost modules for extra power to the PCI-E lanes since my motherboard doesn't have an extra molex connector.

 

[writhziden]

To be honest, your guess is as good as mine. I have never used the configuration you describe, so I cannot really comment on it. There is a graphics cards forum on these forums: Graphic Cards - Windows 7 Forums

There is also a hardware and devices forum: Hardware & Devices - Windows 7 Forums

Those are the two places I would recommend starting a thread (try one forum, and then try the other if the first one does not get your the information you seek; do not start two threads at the same time or the administrators may intervene).

 

[Baasha]

To be honest, your guess is as good as mine. I have never used the configuration you describe, so I cannot really comment on it. There is a graphics cards forum on these forums: Graphic Cards - Windows 7 Forums

There is also a hardware and devices forum: Hardware & Devices - Windows 7 Forums

Those are the two places I would recommend starting a thread (try one forum, and then try the other if the first one does not get your the information you seek; do not start two threads at the same time or the administrators may intervene).

Okay, I'll start a thread in one of those sub-forums.

I removed the motherboard yesterday and reseated EVERYTHING; the CPU, the GPUs, the RAM and reconnected all the other devices. With the 4th GPU, the system does not even POST most of the times. I got it to boot to Windows once out of 10 reboots and even then, the system locked up after about a minute or so.

There is something definitely wrong with the motherboard. I will be calling ASUS first thing tomorrow to RMA it.

Thanks for the help so far. I will update this post once I figure out if the motherboard was indeed the issue.

 

[writhziden]

Alright, look forward to finding out. Sorry you had such trouble.

 

[Baasha]

Alright, look forward to finding out. Sorry you had such trouble.

You're not going to believe this.

Just as a last ditch effort, I figured I'd try reinstalling Windows from scratch.

I cleared the CMOS, reflashed the BIOS (v.1001), secure erased the SSDs, and installed the OS from scratch and the system has been working all evening without a hitch!

The only issue I'm having now is when I installed EVGA Precision X 3.0.2 to benchmark my GPUs, the actual benchmark shows only ONE GPU being used! The other 3 are "load: 0%". Nvidia Control Panel doesn't open either so I'm not sure what's causing that. I have the WHQL 301.42 drivers and after installing that, I enabled 4-Way SLI and OC'd my CPU to the old settings (4.41GHz @ 1.30V). I've restarted the system (after driver updates/programs etc.) at least 25 times and every time it booted right to the desktop without any hiccups; and rather quickly to boot (pun-intended)!

The only thing is this GPU testing is not going well so I need to figure that out. Perhaps SLI is indeed broken on the motherboard but the NVCP, when it worked, showed 4-Way SLI so I'm not sure.

So the question is, why is NVCP not starting/opening up? And, when I ran OC Scanner (w/ Precision X), it showed only one GPU being used. Any ideas on this?

*EDIT: So I installed Unigine Heaven 3.0 (DX11 benchmark) and ran it for a while. It showed ALL four GPUs being used (mostly maxed out @ 99%) so the main issue I'm having now is that NVCP will NOT open no matter what.

 

[writhziden]

I have seen such strange things happen before. Odd that it would cause hardware like symptoms like POST issues, though...

For the nVidia Control Panel problem: Have you tried re-installing all nVidia software? You will want to uninstall all first through Start Menu -> Control Panel -> Uninstall a program. Start with the NVIDIA Graphics Driver (it should remove the rest of the NVIDIA software with it). Restart your computer, and then confirm that Uninstall a program shows no other NVIDIA software. If it does, remove the remaining NVIDIA software, as well. Then install the driver package you want to install fresh.

 

[Baasha]

I have seen such strange things happen before. Odd that it would cause hardware like symptoms like POST issues, though...

For the nVidia Control Panel problem: Have you tried re-installing all nVidia software? You will want to uninstall all first through Start Menu -> Control Panel -> Uninstall a program. Start with the NVIDIA Graphics Driver (it should remove the rest of the NVIDIA software with it). Restart your computer, and then confirm that Uninstall a program shows no other NVIDIA software. If it does, remove the remaining NVIDIA software, as well. Then install the driver package you want to install fresh.

Well, my joy was short-lived.

I had the computer on pretty much all day; installing programs, games, and just doing work. The machine seemed completely fine. I am running the CPU OC'd by the way @ 4.41GHz.

Just to make sure, I wanted to test my rig so I started Trine 2; one of my favorite games.

I maxed out all the settings and fired it up and it seemed to start well. Within 2 minutes of starting the game, the system locked up. I had to hard reset; there was no BSOD. Once I got back into Windows, within 30 seconds, the screen went black and the computer restarted only to show on the Windows desktop that it had a BSOD of 0x1000007e. I've attached the dump files so please let me know what it says.

I still haven't given up hope because this was after installing so many games and programs and everything seemed perfect! I went from not POST'ing to being able to run the rig almost all day without a single hitch.

By the way, during the game, I was OC'ing my cards @ 900MHz/2200MHz.

On the NVCP issue, I just reinstalled the NVIDIA drivers and it came back without issue. I think the problem came about because some program I installed asked to install DirectX and I let it so that caused some conflict with the NVIDIA NVCP. After the reinstallation of the drivers, the issue was resolved.

However, this new BSOD is a whole other issue!

Please help!

 

[writhziden]

First, turn off all overclocking and get the system running stable. Do not overclock until you are 100% sure the system is stable. Then overclock in small steps rather than going straight for the settings you like. After each step, check again that the system is 100% stable.


Also, make sure your RAID drivers are up to date by installing straight from Marvell. Marvell - Support

On that note, RAID was really not designed with the home user in mind. It was designed for companies running servers. Why RAID is (usually) a Terrible Idea - Puget Custom Computers


The crash points to your display card drivers. Do the above first; it may have been due to the overclock...

 

[Baasha]

First, turn off all overclocking and get the system running stable. Do not overclock until you are 100% sure the system is stable. Then overclock in small steps rather than going straight for the settings you like. After each step, check again that the system is 100% stable.

Okay, I will try that out but I'm pretty sure my CPU OC is 100% stable. I ran Prime95 Blend, OCCT, and IBT and they all passed without errors. Anyway, I'll put everything to default and play the game to see if it freezes. I tried it without OC'ing the GPU last night and it froze again. This time, it was worse because I was in the BIOS and the screen went black like before; I think the motherboard is giving up.

Also, make sure your RAID drivers are up to date by installing straight from Marvell. Marvell - Support

That page for the drivers only shows drivers for "Yukon" devices which relates to some "Ethernet Controller" and not the RAID Controller. I downloaded the RAID Controller driver (mv61xx) from another site and they are dated 5/6/2011 but the name is "Generic Marvell RAID Controller". Is there a newer version for those? I can't find them on Marvell's site!

The crash points to your display card drivers. Do the above first; it may have been due to the overclock...

I will set everything to default, reinstall the GPU drivers and see what happens. I just feel like the motherboard is the cause; if it freezes/crashes again, I will be RMA'ing it.

Thanks.

 

[writhziden]

RAID: Marvell* SATA Driver should help.