Solved Fresh Build - Constant BSOD - Multiple Codes - 3b, 50, a

[Vir Gnarus]

Well, it's evident there looks to be some major malfunctioning of the TCP stream. The wireshark logs show a lot more explicit evidence of this. It looks like huge bursts of TCP retransmitting of packets. The first log with the download of Windows update file showed a TCP stream lasting 7 seconds, then a sudden gap of 31 seconds during the TCP stream and then an immediate retransmission of the packets afterwards. Then it goes for another 17 seconds with a few mass retransmissions then there's a 65 second gap and another mass retransmission.

Wireshark is having problems analyzing the TCP stream properly because every single TCP ACK packet appears to have a zeroed checksum. Often this is because of a feature on NICs called Checksum Offloading. You can turn it off by typing "view network connections" in Start Menu then right-clicking and going to Properties for your associated network adapter, then click the "Configure" button and then mosey on over to the Advanced tab. There should be an option called TCP Checksum Offload, which you can then set to disable.

Anyways, I reaaaally doubt we're dealing with some misbehaving program here. You should start by uninstalling and reinstalling your network drivers with the latest available version from your mobo's manufacturer (make sure to download your network drivers first before uninstalling them!). If that does not work, you may need to cough up some dough for a cheap NIC then connect through that and test connectivity. We could very well be dealing with a failing network controller here.

If you wish to make another run of Wireshark with Checksum Offloading disabled, I highly recommend you run both wireshark and xperf simultaneously. Only capture the download before it starts and then stop after it hangs for a bit. Stop the logging before you stop the download.

That should most likely be our final run of logs for this.

 

[LP81]

I re-downloded the latest Drivers directly from Gigabyte's site.
Fully uninstalled the LAN drivers.
Restarted the computer.
Verified there were no network drivers.
Fresh install of LAN Drivers.

Tested file download - stalled out.

Went into the network properties, and I have both TCP Checksum Offload (IPv4) and (IPv6). Options were Disabled, Rx & Tx Enabled, Rx Enabled, Tx Enabled. I chose "disabled" both of them.

Full shutdown and restart to verify setting kept.

Started xperf and Wireshark.

Waited approximately 10 seconds to start the download to ensure both monitoring services were running.

As soon as it stalled out, I stopped both xperf and Wireshark (without killing the download)

Logs can be found here: Zippyshare.com

Note: I did try Safemode + networking and both times, files seemed to come down without incident, but I want to run it a few more times to verify this finding.

 

[Vir Gnarus]

You gave me an xperf log and a Procmon log, not Wireshark? Lol.

EDIT: nm, I forgot both use the same extension for their logs.

 

[LP81]

Oops, it's the right file, but I put the wrong file extension on it when I saved it out.

Just change .PML to .pcap

Sorry about the confusion

Edit: I see you figured it out, I replied based on the email notification

 

[Vir Gnarus]

All right, you can re-enable checksum offloading, otherwise it'll use up your CPU more when doing network activity as opposed to offloading checksum calculations onto the network controller.

I'll go through all this, thanks. I noticed immediately that at the 12-14 second mark on xperf there's a huge burst of net activity, which has a hiccup in the middle of it, and then afterwards drops to nothing for the duration of the log. The xperf log coincides with this as you can see in the attached graphs. You said that you stopped logging as soon as you saw that the download was halted, but it looks here like it continues on for about another 15 seconds or so before the logging stops on either one. Anyways, I'll try to see what I can scrounge up if I can find anything.

Another thing: I noticed in the xperf logs you've given me that for some reason the stacks have not been saved, (the 'stackwalk' feature), which kinda leaves me out in the dark a good bit on what's going on. You made sure to do the registry tweak by adding DisablePagingExecutive, correct? Can you verify it has been entered correctly? Any misnomer (note that registry keys can be case sensitive, I think) can prevent it from working properly.

 

[LP81]

When I ran this, it didn't seem to do anything:
REG ADD "HKLM\System\CurrentControlSet\Control\Session Manager\Memory Management" -v DisablePagingExecutive -d 0x1 -t REG_DWORD -f

Because when I tried to run "xperf -on latency -stackwalk profile" it gave me a warning that the stackwalk wasn't enabled

So I manually changed the value to 1 in regedit

See attached image.

It no longer gave me the warning when running xperf

Please let me know if I missed something or skipped a step

 

[Vir Gnarus]

Yes, that's splendid. You'll need to restart and do - sigh - another round of logging the same as before. Make sure to have a stopwatch or something in hand to record when you start the download and when you see it dropping to nothing. Of course, make sure there's no error message about the stackwalk thing.

 

[LP81]

To clarify, that registry setting was like the when I ran this last batch of tests...

The screenshot was merely showing it was already done.

I can restart again and run the test if you like. I have my running watch which will do split times.

 

[Vir Gnarus]

So you did not receive the stackwalk error last time you ran the tests? That sounds odd given that it's not saving the stacks as it's intended to do.

 

[LP81]

Correct - see attached screenshot of the CMD

No errors

 

[LP81]

So, while going the the "Advanced" options on the LAN connection to turn off the TCP Checksum, I noticed there are a TON of options to play with and was wondering if any of them are worthwhile to explore toggling. Screenshot of all of the options below. I'm thinking the "Energy Efficient Ethernet", "Flow Control" or "Green Ethernet" might be options to pursue if the logs don't reveal anything in particular.

 

[Vir Gnarus]

Since this isn't on a laptop, all the energy saving options you can consider disabling.

As for the logs missing their callstacks, I found what the deal was. I was checking the summary table on the wrong graph. Go figures. Lol.

Anyways I'll see what I can pull up.

 

[Vir Gnarus]

This problem is posing itself to be extremely elusive. The gap in the center of the TCP stream (downloading) can be explained by a full TCP window and therefore stuff got corrupted and resending needed to take place. But I do not see that issue recurring when the download stalls completely. There's really no explanation. It just kinda... dies. Suddenly.

The Xperf log doesn't seem to reveal anything suspicious either. I don't see some thread or process start up when the download stalls, so there's nothing really I can correlate there. The stackwalks so far do not reveal anything suspicious.

I am still looking into this right now. The only things I can remotely consider suspect here is window scaling, LogMeIn, and your antivirus software. The latter two can be dealt with by disabling or removing them, but that sounds like an ugly solution when I can only faintly suspect them with the evidence I have right now. Still worth a shot. The window scaling problem sounds more likely.

Window scaling can occur on the router, the network adapter, and Windows itself. If either does not support scaling well (most likely the router or adapter) then things can get ugly. You can go to the properties for your network adapter like you did before and turn off the "Receive Side Scaling" option. That'll turn it off for the adapter. For Windows - which calls it "auto-tuning" - you can disable it (and subsequently reenable it later if need be) by following instructions by canspec here.

Also, window scaling issues can occur commonly when using older routers, or routers with outdated firmware. Make sure to update the firmware on your router to the latest. Chipset drivers and BIOS should also be updated since they can affect this.

If you plan on logging again for whatever reason, turn off all Offloading options on your network adapter (TCP/UDP/IPv4). In addition, having a log of downloading the same file (preferably only a snippet at the beginning if the file downloads ok) using one of your other working computers, preferably when attached to the same port of the router as your PC. This will give a good solid baseline on what I should expect to see and any differentials I may discover. Make sure to run it just as before, only stopping about 20-30 seconds in if the download continues ok. Provide both xperf and wireshark logs as previously instructed.

I will still continue to analyze the data as I haven't gone through it all yet, and there may be something lurking within. Any more data from a working PC would be especially beneficial, though.

 

[LP81]

I have turned off Energy Efficient Ethernet and Green Ethernet

NO changes

However one little quirk...

If I log into my computer remotely via LogMeIn and do the same downloads, they don't seem to time out at all.

I will try the auto-tuning of the Card and windows next to see if that helps anything.

Router & Modem aren't all that old, but I'll check for firmware updates - using an Cisco RVS4000 4-Port Gigabit Security Router and a Motorola Surfboard Modem - SB6120 (not a cheapo rented Comcast box)

I wish I could update the BIOS on my Mobo, but all of them on gigabyte's site are for 32-bit systems and it won't let me run the update (and i really don't want to install their bloatware)

 

[LP81]

Hows this for a monkey wrench

I have confirmed that while I have an active Log Me In connection to my computer, my connection is near flawless...

I was able to watch YouTube Videos, download files and play Diablo III with no connection errors

 

[Dave76]

Don't mean to interrupt Vir Gnarus work here, just wanted to let you know BIOS is independent of your OS being 32/64 bit, the BIOS listed on the page linked below are for 32 & 64 OS.
BIOS loads before your OS starts, therefore it doesn't matter if you have 32 or 64 bit.

Which BIOS version do you have now?
Latest for your board is F8 2012/03/21

*IMHO, uninstall Log Me In, see if your connection is stable.
If you have the same issues then re-install the latest version, if no more issues then...

 

[Vir Gnarus]

Wow, call me bamboozled. I'd like to see what happens when you have LogMeIn uninstalled (make sure the virtual network adapter it installs also goes away).

I can only guess that this can happen if you prioritize LogMeIn's net adapter above the rest for connectivity. Go back to the list of network adapters, then press Alt, then go to Advanced > Advanced Settings. Look at the order for Connections, Bindings for Local Area Connection, and Provider Order to verify the prioritized order of things. LogMeIn should be around last in line (bottom) for Connections, and if it has any other entries in Provider Order and Bindings, make sure those are remedied as well. Of course, if you happen to already have installed LogMeIn, it shouldn't show up in any of them.

If you also for some odd reason have any of the connections bridged with LogMeIn's virtual network adapter, that too could pose issues.

 

[LP81]

The stalling started occurring the moment Win 7 was loaded, before LogMeIn, Firefox, MSE, or any program what so ever was installed.

Would it be worth while to try and install from scratch again given the OS was installed prior to all of the hardware RMAs? Despite them being identical hardware, they really are all kinda new: Mobo, processor, RAM (Obviously different type of RAM)

I will test by removing LogMeIn right now but I can guess it isn't going to change anything.

 

[LP81]

Confirmed it's not LogMeIn

Uninstalled LogMeIn
Verified a single Network adapter is present
Restarted Machine
Started a download, crapped out after 5-6 seconds
Tested the pause/resume with usual results.

Note, I have Rev 2.0 of the Mobo, not Rev 1.0. I am on BIOS FB, their most current is FD. Looks like the only way to update my BIOS is via their bloatware since I am running x64 based and it always tells me:

--------------------------- Unsupported 16-Bit Application ---------------------------
The program or feature "...\Win 7 Installs - 5-10-12\bios\FLASHSPI.EXE"
cannot start or run due to incompatibility with 64-bit versions of Windows. Please contact
the software vendor to ask if a 64-bit Windows compatible version is available.

 

[Vir Gnarus]

If you said it was doing this even before you installed all those programs I don't see how that'll work. I mean, you can still try by installing a copy of Windows and testing it there (no need to overwrite your existing Windows installation).

If for whatever reason it does work on the other install of Windows, I'd really like to see an xperf/wireshark log combo on it to juxtapose it with the ones we've already garnered to see just what the difference is. Again, without a baseline, it's extremely difficult to tell what I'm looking for without prior knowledge of the code base for everything.