BSOD Win 7 Pro 32bit VLK Image BUGCHECK_STR: 0xC5_2

[huskietech]

Hi, I am an technician for a company who has windows 7 VLK licensing. I create windows images and deploy them with our deployment server. Recently I have had to undertake imaging a 32bit windows 7 pro image because we have an application that will only work on 32bit. The applications currently work on windows XP Pro 64bit but will not work on windows 7 64bit, but do in fact work on windows 7 32 bit.

I have created the image from scratch and have installed all patches, software, anti-virus and customized the image to a point where I have captured and tried deploying for testing. I have never had an issue with windows 7 64 bit and capturing and deploying my images. I had always injected all drivers needed for the different hard ware we have.

My first approach with the windows 7 32 bit image was the same that I would have done with windows 7 64 bit. I injected all drivers (video, audio, nic, chipset) into the .wim. I made sure to only download and inject 32bit drivers. When I first deployed the image I noticed that I was getting minidump crash files even though a BSOD was not occuring (the actual blue screen memory crash dump did never occur) but when I would sign into windows I would see a bugcheck window and a minidump file was present on the computer. Its almost as if it failed somewhere, but nothing actually crashed that you can see.

I decided to create a new image but this time instead of injecting any drivers into the image I left it as is with only windows updates. I took that image and began to test deployment again, this time thinking that maybe one of the 32bit drivers was corrupted and was causing the problem. Unfortunately it looks like that is not the case, even with all default windows update drivers the system appears to still be generating minidump crash files.

I am usually pretty good at deciphering the crash files myself but I am lost as of now. When I create the image from scratch I do not have any errors or crashes. It seems to be only when I try to deploy the image, again I have never had an issue with imaging and capturing windows 7 64 bit but now I am having an issue with windows 7 32bit.

I have set up my deployment server to capture via x86 with a boot32.wim (86x boot file), also I have changed all of my unattended files to be x86 from amd64. I don't believe I have overlooked changing anything in the process going from 64bit to 32bit in the building of the image and capturing and deploying. It all works, but the end result is a PC that generates crash files.

If someone is able to help me figure out if there is something I should be putting as far as a driver into my image to prevent this from occurring I would greatly appreciate the feedback.

I am in crunch time as there is less than 30 days left I have to finish this image and deploy it to about 50-60 computers.

Thank you for taking the time to read everything, please let me know if there is any information you need and I will provide it. I have already collected the files from the computer needed but I would feel more comfortable PM'ing it to someone since I am unsure if there is anything pertinent to my employer in the logs that I would not want to post publicly. I have attached just the dmp file.

 

[derekimo]

Hi there,

Since it was just a dump file you uploaded I'll just point out the driver I see being flagged,

Unable to load image \SystemRoot\system32\Drivers\SEP\0C010BB9\00A5.105\x86\SYMEFA.SYS, Win32 error 0n2
*** WARNING: Unable to verify timestamp for SYMEFA.SYS
*** ERROR: Module load completed but symbols could not be loaded for SYMEFA.SYS
 SYMEFA+0x1e3a

Driver Reference Table - SYMEFA.SYS

Which is related to Enterprise Support - Symantec Corp.

start    end        module name
8ba87000 8bb6f000   SYMEFA   T (no symbols)           
    Loaded symbol image file: SYMEFA.SYS
    Image path: \SystemRoot\system32\Drivers\SEP\0C010BB9\00A5.105\x86\SYMEFA.SYS
    Image name: SYMEFA.SYS
    Timestamp:        [COLOR="Red"]Fri Jan 18 16:30:44 2013[/COLOR] (50F9E934)
    CheckSum:         000E8F33
    ImageSize:        000E8000
    Translations:     0000.04b0 0000.04e4 0409.04b0 0409.04e4

You can try updating that.

DRIVER_CORRUPTED_EXPOOL [B][COLOR="Blue"](c5)[/COLOR][/B]
An attempt was made to access a pageable (or completely invalid) address at an
interrupt request level (IRQL) that is too high.  This is
caused by drivers that have corrupted the system pool.  Run the driver
verifier against any new (or suspect) drivers, and if that doesn't turn up
the culprit, then use gflags to enable special pool.
Arguments:
Arg1: 00000004, memory referenced
Arg2: 00000002, IRQL
Arg3: 00000001, value 0 = read operation, 1 = write operation
Arg4: 82b3b943, address which referenced memory

Bug Check 0xC5: DRIVER_CORRUPTED_EXPOOL (Windows Debuggers)

The kernel attempted to access pageable memory (or perhaps completely invalid memory) when the IRQL was too high. The ultimate cause of this problem is almost certainly a driver that has corrupted the system pool.

If you get no help from that try enabling Driver Verifier,

   Information

Run Driver Verifier for 24 hours or the occurrence of the next crash, whichever is earlier.
Driver Verifier - Enable and Disable

Driver Verifier will cause your computer to run very sluggishly - this is normal. What it is trying to do is force your system to BSOD and isolate the offending driver/s. When it does, reboot, disable driver verifier, reboot as normal and upload the new dmp file/s here.

I recommend creating a system restore point before turning on driver verifier:
System Restore Point - Create

If your system fails to boot to desktop once driver verifier is enabled, turn it off by booting into Safe Mode:
Safe Mode

As for the other info you are worried about, there really is no personal info in the logs.

 

[huskietech]

When I first had the BSOD I was able to find something on Symantec being the potential problem as well. I was doubtful because it has worked flawlessly with all of my 64bit images, and thought it may have been all of the hardware drivers I injected into the image (again I have always done this for 64bit images and have never had a problem).

I am uploading the rest of the logs captured on the computer encase anyone else is able to give some insight. I followed the link you provided but the driver page just links to Symantec. I found a post but it doesn't make sense, I am not able to delete the efaData folder required to rebuild the Symantec database created that is causing the crashes.
https://community.norton.com/t5/Nor...sh-logs-point-to-SYMEFA-SYS/td-p/38501/page/6.


We use SEP version 12.1 and in order to avoid duplicate ID's I run this utility before running sysprep on the computers so that when the images are deployed a new managed SEP client will be created, again I run this on all 64bit images and have never had a problem but it almost seems that when windows loads and it tries to start the Symantec services again for the first time that it is crashing. I can try to deploy the image without Symantec and just install it afterwards but I would really like to try to keep it within the image.
http://www.symantec.com/business/support/index?page=content&id=TECH163349


I will continue to try to investigate Symantec's website and use Google to find out how to delete the folder specified to eliminate the BSOD.


Also when I tried to run driver verifier before with an image that contained all hardware drivers the system would not even boot past the windows logo so I went back to the drawing board and created an image with ONLY windows update drivers, none from HP and now I am still getting the BSOD attached.

*I contacted Symantec endpoint support and they suggested that I install the latest version 12.1.4 (I don't see how there could be a variance from 64bit windows where I have never had an issue with 12.1.3 and now need to install the newest version with my 32bit image).

 

[derekimo]

Usually we suggest to uninstall an offending AV product when it is being flagged as the cause, and install Microsoft Security Essentials because it is known to not cause BSOD's.

Due to the situation you are describing, it sounds like that is not an option so I can only suggest following up with Symantec.

You can see from the info I posted above, that BugCheck is driver related and so far the only driver being flagged is the one mentioned.

If you want to try uninstalling Symantec for testing purposes and see if it still crashes we can take it from there.

 

[huskietech]

Thanks for the reply and your help earlier finding the problematic application. I have already uninstalled Symantec 12.1.3 and installed the newest version 12.1.4a according to their tech support. I have also re-captured the image and just finished re-deploying it. I am crossing my fingers that it does not crash because I have ~80 in place upgrades to do within the next 4 weeks.

 

[derekimo]

You're welcome, Hope that does the trick for you, looks like you are going to be busy.

Let us know how it goes.

 

[huskietech]

I have a question regarding my initial .dmp files that the computer was giving. Originally I captured my 32bit image and injected video/audio/network/chipset drivers for 4 models of HP computers (this is something I never had an issue with using 64bit). I deployed this image and was getting BSOD, I figured it was a driver issue regarding the packages I had added. I then started over but this time I did not inject any drivers, I still had the BSOD from above so here I am thinking I can again inject my driver packages for deployment.

Would you mind taking a look at these 2 .dmp files from my initial attempt at imaging with 32bit, I don't know if they are hardware driver related or if they are also dump files pertaining to my Symantec software issue.

Thanks,

 

[derekimo]

Sure, I don't mind.

This is from the dump dated 03-03-2014,

The same driver,

Unable to load image \SystemRoot\system32\Drivers\SEP\0C010BB9\00A5.105\x86\SYMEFA.SYS, Win32 error 0n2
*** WARNING: Unable to verify timestamp for SYMEFA.SYS
*** ERROR: Module load completed but symbols could not be loaded for SYMEFA.SYS
 SYMEFA+0x1e3a

The other one from the dump dated 03-10-2014, earlier in the day from your original dump,

BugCheck C5, {4, 2, 1, 82b58943}

Probably caused by : Pool_Corruption ( nt!ExDeferredFreePool+2e3 )

Followup: Pool_corruption

Is another C5 bugcheck but no specific driver is pointed at.

There are some mentions of a usbhub,

Win32 Start Address nt!PopIrpWorker (0x82a4cb28)
Stack Init 91135fd0 Current 91135bf8 Base 91136000 Limit 91133000 Call 0
Priority 13 BasePriority 13 UnusualBoost 0 ForegroundBoost 0 IoPriority 2 PagePriority 5
ChildEBP RetAddr  Args to Child              
9113570c 82b58943 badb0d00 82b6d7f8 00000001 nt!KiTrap0E+0x1b3 (FPO: [0,0] TrapFrame @ 9113570c)
911357b8 82b5835f 82b6d6c0 00000000 82b57aba nt!ExDeferredFreePool+0x2e3
91135824 92a3d3f2 85f17008 70627375 82a71788 nt!ExFreePoolWithTag+0x8a4
9113585c 92a46aeb 8810ff68 86408378 88b465b8 USBPORT!USBPORT_Core_iCompleteDoneTransfer+0x7c5 (FPO: [Non-Fpo])
91135888 92a4691b 86b1c000 00000000 86b1c718 USBPORT!USBPORT_Rh_CtrlEp_Worker+0x19d (FPO: [Non-Fpo])
911358ac 92a4031f 86b1c028 87dfa008 86408378 USBPORT!USBPORT_RH_NeoQueueAsyncTransfer+0x8c (FPO: [Non-Fpo])
911358d4 92a4621d 86b1c028 86b1c028 00000000 USBPORT!USBPORT_Core_QueueTransferUrb+0x143 (FPO: [Non-Fpo])
911358e8 92a3f088 86b1c028 86408378 88b465b8 USBPORT!USBPORT_ControlTransfer+0x6c (FPO: [Non-Fpo])
91135938 92a3f52c 871b8028 00000000 86408378 USBPORT!USBPORT_ProcessURB+0x752 (FPO: [Non-Fpo])
91135960 92a3ca34 871b8028 86408378 8810ff68 USBPORT!USBPORT_PdoInternalDeviceControlIrp+0xfb (FPO: [Non-Fpo])
91135988 82a6dc1e 871b8028 871b82cc 88b465b8 USBPORT!USBPORT_Dispatch+0x18a (FPO: [Non-Fpo])
911359a0 94c1c6dc 91135a70 00000002 87f8e028 nt!IofCallDriver+0x63
911359d4 94c1c815 87f8e028 91135a0c 91135a00 usbhub!UsbhSyncSendCommand+0x197 (FPO: [Non-Fpo])
91135a18 94c1ee35 87f8e028 00000002 91135a70 usbhub!UsbhQueryPortState+0xcc (FPO: [Non-Fpo])
91135a68 94c1f859 00000000 871ae9d8 871ae9ec usbhub!UsbhHubRunPortChangeQueue+0x105 (FPO: [Non-Fpo])
91135a8c 94c1fb20 87f8e000 00000005 871ae9ec usbhub!Usbh_PCE_wRun_Action+0x124 (FPO: [Non-Fpo])
91135aac 94c200fd 87f8e028 871ae9d8 00000005 usbhub!UsbhDispatch_PortChangeQueueEventEx+0xb9 (FPO: [Non-Fpo])
91135ad8 94c200d0 87f8e028 871ae9d8 00000005 usbhub!UsbhDispatch_PortChangeQueueEvent+0x24 (FPO: [Non-Fpo])
91135b08 94c20095 87f8e028 871ae9d8 00000005 usbhub!UsbhDispatch_PortChangeQueueNullEvent+0x20 (FPO: [Non-Fpo])
91135b24 94c30034 87f8e028 871ae9ec 871ae9d8 usbhub!UsbhPCE_wRun+0x48 (FPO: [Non-Fpo])
91135b6c 94c206f0 87f8e028 871aeb60 862c80b0 usbhub!UsbhWaitEventWithTimeoutEx+0x15b (FPO: [Non-Fpo])
91135b98 94c1fad8 87f8e000 00000004 87f8e5d8 usbhub!Usbh_PCE_Disable_Action+0x284 (FPO: [Non-Fpo])
91135bb8 94c203dc 87f8e028 871ae9d8 00000002 usbhub!UsbhDispatch_PortChangeQueueEventEx+0x71 (FPO: [Non-Fpo])
91135bf8 94c1ce6c 87f8e028 00000002 87f8e5d8 usbhub!UsbhPCE_Disable+0x78 (FPO: [Non-Fpo])
91135c1c 94c1cd38 00000002 87f8e5d8 87f8e028 usbhub!UsbhBusPause_Action+0xf5 (FPO: [Non-Fpo])
91135c38 94c1dd05 00000003 87f8e5d8 00000007 usbhub!Usbh_BS_BusRun+0x6a (FPO: [Non-Fpo])
91135c54 94c202ca 00000003 87f8e5d8 00000007 usbhub!UsbhDispatch_BusEvent+0xcb (FPO: [Non-Fpo])
91135c74 94c34f3f 87f8e028 87f8e5d8 00000003 usbhub!UsbhSyncBusPause+0x38 (FPO: [Non-Fpo])
91135ca0 94c31ae5 8626cedc 88a116b8 8626ce00 usbhub!UsbhFdoSetPowerDx_Action+0x51 (FPO: [Non-Fpo])
91135cc8 94c310de 87f8e5d8 00000004 87f8e028 usbhub!UsbhFdoDevicePowerState+0x189 (FPO: [Non-Fpo])
91135ce4 94c1bbab 87f8e028 87f8e0e0 82b78c40 usbhub!UsbhFdoPower_SetPower+0x6d (FPO: [Non-Fpo])
91135cf8 82a4ce79 87f8e028 8626ce00 00000000 usbhub!UsbhGenDispatch+0x63 (FPO: [Non-Fpo])
91135d50 82c4013d 85b27bc8 bd1b449d 00000000 nt!PopIrpWorker+0x351
91135d90 82ae7559 82a4cb28 85b27bc8 00000000 nt!PspSystemThreadStartup+0x9e
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x19

That may be something for you to look at.

 

[huskietech]

I had a feeling the Symantec one was going to show up again, but I am a little thrown off by the usbhub crash. With the current HP computers we have their is bad hardware/driver issue with the USB 3.0 drivers that on its own caused frequent BSOD crashes so now I don't include that driver ever.

I'll have to test to see if I get any BSOD within the next 24hours with just the newer version of Symantec, as well I am currently adding drivers to a copy of my .wim to test re-deploying it with all drivers as well as the new version of SEP.

Thanks again for taking the time to get back to me.

 

[derekimo]

You're welcome, those mentions of usbhub may not be the actual cause, I just didn't see any other info in that dump.

Driver Reference Table - USBPORT.SYS

Driver Reference Table - usbhub.sys

Those are both Windows drivers so most likely not the true cause.

I assume your images are updated before deploying them.

 

[huskietech]

You are correct, images are first fully updated by windows and than company applications are installed afterwards, HP drivers are then downloaded and added to the baseline image so whatever hardware the image is installed on the appropriate drivers will be selected and used (this has been my process and has never failed me). I'm hoping the issue stems from Symantec and is going to be resolved after updating to the newest version.

Thanks again!

 

[derekimo]

You're welcome. Let us know how it goes.

 

[huskietech]

I thought I had my BSOD under control because I tested the image without drivers added and with just upgraded Symantec. I guess I got cocky because it was fine testing for a day so I added my driver packages today for all hardware and deployed the image, now I have BSOD on two different hardware computers.
The .dmp files are attached, any ideas?
thanks,

 

[derekimo]

Those are both the same Bugcheck C5,

The earlier one is blaming Symantec again,

Unable to load image \SystemRoot\system32\Drivers\SEP\0C010FAD\0FAD.105\x86\SYMEFA.SYS, Win32 error 0n2
*** WARNING: Unable to verify timestamp for SYMEFA.SYS
*** ERROR: Module load completed but symbols could not be loaded for SYMEFA.SYS
 SYMEFA+0x1e3a

The other one wasn't flagging a specific driver, just pool corruption.

BugCheck C5, {4, 2, 1, 82d27943}

Probably caused by : Pool_Corruption ( nt!ExDeferredFreePool+2e3 )

Followup: Pool_corruption

 

[huskietech]

Thanks again for checking those, how are you able to get the below from the .dmp files? when I run WinDbg I don't ever see that, again I'm not a pro at system debugging by any means, usually I just press the analyze button and then if I'm lucky it will tell me a specific driver at fault, or in this case it just keeps giving me c5 bugcheck which all seems to point to Symantec and 32bit windows. Very Frustrating, it appears to be maybe the capturing of the image with Symantec and re-deployment of the software that sometimes it crashes on a clean deployment. I have two image files, one seems to work, and another with all my drivers installed that gives the BSOD. Very very annoying especially when I need to get the ball rolling on deployment. I guess I will use the image that doesn't crash and install hardware drivers after the fact.

"Unable to load image \SystemRoot\system32\Drivers\SEP\0C010FAD\0FAD.105\x86\SYMEFA.SYS, Win32 error 0n2
*** WARNING: Unable to verify timestamp for SYMEFA.SYS
*** ERROR: Module load completed but symbols could not be loaded for SYMEFA.SYS
SYMEFA+0x1e3a"

 

[derekimo]

Try using this method,

http://www.sevenforums.com/crash-lockup-debug-how/277355-debugging-bsod-my-way.html#post2329908

That's how I got that info about SYMEFA.SYS

When you get to step 2, it's dps (limit) (base) enter.

 

[huskietech]

thank you for all your help with my problem, I just tried using that !thread myself and was able to see what you were referring to. Very nice! it will be awesome to have this going forward for bugging and troubleshooting things.

 

[derekimo]

You're welcome, good luck, hope you get that figured out.

 

[huskietech]

I've got a new one on a machine that was BSOD win windows XP 64x, now re-imaged with windows 7 x86 and it is still BSOD. Any ideas from this .dmp file? I could not see anything.

 

[huskietech]

Hi again, I was hoping you could troubleshoot this log with me derekimo; See attached.

I was unable to find a driver issue after running the !thread and dps commands. Apparently the user just launched IE and the next thing the computer BSOD crashed. Any ideas?

Thanks so much!