Multisytem Dell 8300 BSODs

Weird multi system BSOD


Have a bunch of Dell Dimensioin 8300's we purchased (Win 7 Enterprise, 64 bit, Purchases 1/2011). Been putting the same image on them for past year. All of a sudden that image is throwing some BSOD. Throws them pretty quick, within 30 minutes.

Ok, no big deal, something slightly different in the hardware in one of the batches from Dell that the image doesnt like. So I reformat a drive, install a clean new copy of Windows 7 Enterprise 64 bit. After installing lal the drivers for that specific Dell the BSOD comes back. So I install a new clean copy of Windows 7 64 bit and begin installing drivers one by one with break in between waiting for BSOD.

Let it sit for few hours after clean install. No BSOD.

I install each missing device manager missing component, and let it sit for a few hours.
Chipset controller 1: No BSOD after 3 hours
Chipset controller 2: No BSOD after 3 hours
Broadcom Net xTreme NIC contorller: No inital BSOD but get a BSOD after using the newly installed NIC to access the internet.

This is the recommended NIC driver for this machine from Dell.

Can someone look at this dmp file and see if it is really NIC-related?

And I wouldnt mind learnign to interpret a BSOD myself if there are some simple tutorials.

Many thanks!

Do you only have one crashdump? I'm not sure I can use this crashdump as it's a system service exception and therefore any related thread other than the service thread will not be retained in the minidump. In short, it just means the minidump doesn't have enough info to make a firm suspicion on anything.

I do have a couple of clues though. The running process at the time was TrustedInstall which is pretty selfexplanatory. McAfee filter drivers were also present in the running thread's stack, of which they are dated woefully back in January 2011. Not sure if you can update them. They may not be responsible, however. Again, no solid evidence.

I checked the syslog, and I found what appears to be isolated error events that occurred only once, one (Windows update) 2 minutes prior to the events related to the crash (which occur during startup) and one that's 6 minutes behind. They are as followed:

Code:

Event[461]:
  Log Name: System
  Source: Microsoft-Windows-WindowsUpdateClient
  Date: 2012-04-10T11:18:28.049
  Event ID: 20
  Task: Windows Update Agent
  Level: Error
  Opcode: Installation
  Keyword: Installation,Failure
  User: S-1-5-18
  User Name: NT AUTHORITY\SYSTEM
  Computer: em-wohlfeil
  Description: 
Installation Failure: Windows failed to install the following update with error 0x80070643: Update for Windows 7 for x64-based Systems (KB979538).

...

Event[563]:
  Log Name: System
  Source: Microsoft-Windows-Directory-Services-SAM
  Date: 2012-04-10T11:13:56.817
  Event ID: 12291
  Task: N/A
  Level: Error
  Opcode: Info
  Keyword: N/A
  User: S-1-5-18
  User Name: NT AUTHORITY\SYSTEM
  Computer: em-wohlfeil
  Description: 
SAM failed to start the TCP/IP or SPX/IPX listening thread

I'm not entirely sure what to do with this knowledge though you may know more about it. If you have any logs from other systems I'd love to see them for to determine patterns related to them all.

Are you sure this is the recommended driver for this installation? The driver shows ups as being dated from June 2010, which is quite old. If need be, you may need to contact Dell support about it as they might have hotfix versions of the driver sitting in their internal repository that they will distribute on a need-too basis.

If you'd like, on this test system you can startup Driver Verifier to see if it'll shed more light on this. Note that it's designed to crash the system upon detection of driver bugs, so this test system BSODs again after starting DV, provide us any new crashdumps.

Thanks for the reply! This is the only one I have, for now. I had several other on a different hard drive I had in this machine. Tried a couple different new hard drives trying to rule out the HD, but reformatted them reinstalling the OS losing those dumps.

The McAfee stack was probably running at time of Bluescreen. But I suspect that was because I had just installed the NIC driver and then immediately installed McAfee. And the last thing McAfee install does is go out and update itself - which used the NIC for the first time. And I guess that the NIC, being used/taxed for the first time, caused a BSOD while McAfee was updating - so when it crashed McAfee was runnning and would show errors in logs. This is an insitutional verison of McAfee that is having no issue on other machines (some recently installed). Will check with the insitution for any updates.

Its something specific to this batch of Dell 8300's. I downloaded the Broadcom NIC driver direct from Broadcom and its currently running 14.8.2 from 7/12/2011 whereas the one from the Dell site for this machine is 14.2.0.7 from 1/9/2011. One hour and no BSOD yet.

Will run it and put it thru its paces over next 24 hours and try to get it to bluescreen - or not. Will also run drive verifier now. Thanks agian and I will keep you updated. Almost wish it would BSoD so I could learn from this - navermind, I take that back, happy its running.

You said the broadcom driver provided by Dell was dated 1/9/2011? Because the one I saw in the dump file being used was dated June 8, 2010.

That is odd. When I referenced 1/9/2011 that was the release date on the Dell site at the driver's download page:
Drivers and Downloads | Dell [United States]

After fresh OS install the Device Manager showed no NIC, I ran that executable downloaded from Dell and just assumed its refence to a 1/9/2011 release date meant it was the driver version date. Setup reportrd it installed fine and NIC began working, and Device Manager was happy. Wonder if the 1/9/2011 release date contained a 6/8/10 driver ?!??! Sounds like Dell.

This is getting wierder and wiered. I finally started getting BSod's again. Attached is the zip file with the dumps and other fiels. Seems very hardware related with a lot of BUGCHECK_STR: 0x124_GenuineIntel. Again the wierd part being happening on a "batch" of PCs from Dell, even with new OS installed.

Yeah, a good majority of them report an internal timer error. This was during core 3 use when attempting to access bank 3 of the internal CPU cache. The fact that they are extremely consistent kinda leads me to believe we still got some software issues here. While these WHEA errors are referring to a CPU problem, I've seen software like that which is included with motherboards to cause these.

Have you tried turning on Driver Verifier yet? Note that when you do so, when you're at the part to select which checks to use, follow the instructions mentioned in the article I linked too previously but also in addition do not check Force Pending I/O Requests. Send any minidumps for any crashes that should occur after DV is turned on.

Btw, have you updated your BIOS and chipset drivers as well? These can commonly cause issues that manifest themselves as WHEA errors. Any overclocked state will also do it.

I personally would recommend contacting Dell about it. If you have several systems causing the same problem even when reinstalling the OS and updating drivers, then you may have a bad batch of PCs or there's an underlying problem with that specific Dell model that's causing this. Eventually it may end up having you resort to going to Dell anyways so it's best to do so an try and escalate your situation to a higher level tech support so they can work with you on it.

Thanks for the reply. I did run Drive Verifier and it crashed. Second time it finished but couldnt locate the report, if any. Will re-read those steps.

Updated Bios and ire-installed chipsets (there were 2). But those were from Dell's driver list for these systems, not necessarily any update from Intel's site. Will gather more info! The thought of calling Dell, let alone trying to esclaate, make me mauseius

Driver Verifier will cause the system to crash when it detects an issue, and hopefully a crashdump will be produced as a result. You can send us any resulting minidumps that may have occurred when Driver Verifier was active.

Yes, I know calling Dell is a last-ditch effort, but some cases like this it's a necessary one, since it sounds a lot more to be their problem than yours.