Code:
BugCheck 101, {[COLOR="Red"]19[/COLOR], 0, [COLOR="Green"]fffff88003517180[/COLOR], [COLOR="Blue"]6[/COLOR]}
This bugcheck indicates that a clock interrupt failed to synchronize all the processors within the allocated time interval which in this case was
19 ticks.
So what is a clock interrupt?
Well a clock interrupt is a synchronization mechanism that calls all the processors and asks them to call in within the allocated time interval, when one doesn't respond the system bugchecks.
The 3rd parameter is the PRCB address for the faulting processor that didn't respond to the clock interrupt.
Which we shall get around to soon.
Most of the time the 4th parameter is the faulting processor which in this case is processor #6.
To keep this simple for you the problem appears to come from your Realtek Ethernet Controller.
Code:
6: kd> [COLOR="Green"]lmvm rt64win7[/COLOR]
start end module name
fffff880`054dc000 fffff880`055c3000 Rt64win7 (no symbols)
Loaded symbol image file: Rt64win7.sys
Image path: \SystemRoot\system32\DRIVERS\Rt64win7.sys
Image name: Rt64win7.sys
Timestamp: [COLOR="Red"]Wed Apr 23 03:11:19 2014[/COLOR] (53572147)
CheckSum: 000E7D7B
ImageSize: 000E7000
Translations: 0000.04b0 0000.04e4 0409.04b0 0409.04e4
You could try removing it and letting windows install the driver for you, see if that makes a difference.
If not try an older version, I've seen a couple of issues with this latest version, whether it's a coincidence or not I don't know.
Realtek
Here's my analysis which you don't have to read all of this but I thought I'd do one anyway.
Code:
4: kd> [COLOR="Green"]!prcb 6[/COLOR]
PRCB for Processor 6 at [COLOR="Red"]fffff88003517180[/COLOR]:
Current IRQL -- 0
Threads-- Current fffff880035220c0 Next fffffa8007a20b50 Idle fffff880035220c0
Processor Index 6 Number (0, 6) GroupSetMember 40
Interrupt Count -- 001d49e6
Times -- Dpc 00000087 Interrupt 00000028
Kernel 0001791a User 0000216d
That matches the 3rd parameter so we can confirm that's the faulty processor.
Lets take a look at the callstack and trap frames for the primary processor (#0)
Code:
0: kd> kv
Child-SP RetAddr : Args to Child : Call Site
fffff880`0e49aca8 fffff800`02f18a4a : 00000000`00000101 00000000`00000019 00000000`00000000 fffff880`03517180 : nt!KeBugCheckEx
fffff880`0e49acb0 fffff800`02ecb6f7 : 00000000`00000000 fffff800`00000006 00000000`00002710 fffff880`00000002 : nt! ?? ::FNODOBFM::`string'+0x4e3e
fffff880`0e49ad40 fffff800`02e0d895 : fffff800`02e33460 fffff880`0e49aef0 fffff800`02e33460 fffffa80`00000000 : nt!KeUpdateSystemTime+0x377
fffff880`0e49ae40 fffff800`02ebe113 : 00000000`37f3b484 fffffa80`00f0e580 ffff0000`0fe69fd9 fffff880`054e2221 : hal!HalpHpetClockInterrupt+0x8d
fffff880`0e49ae70 fffff800`02ef74b7 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiInterruptDispatchNoLock+0x163 (TrapFrame @ [COLOR="Green"]fffff880`0e49ae70[/COLOR])
fffff880`0e49b000 fffff800`02ef63cd : 00000000`00000000 00000000`00000001 fa80075b`75c004c0 fffffa80`07722980 : nt!KxFlushEntireTb+0xc7
fffff880`0e49b040 fffff800`02f1a9b0 : 00000000`00000001 fffff8a0`0eb2ca88 00000000`00000001 fffff6fc`c00e3288 : nt!KeFlushTb+0x119
fffff880`0e49b0c0 fffff800`02ed77b4 : 00000000`00000006 fffff880`0e49b140 fffff880`0e49b140 fffff6fc`c00e3288 : nt! ?? ::FNODOBFM::`string'+0xada2
fffff880`0e49b100 fffff800`02ee6bd4 : fffff980`1c651000 00000000`00000001 00000000`00000001 fffff800`02eeafcf : nt!MmCheckCachedPageStates+0x734
fffff880`0e49b2b0 fffff800`02ee6050 : fffffa80`07a72380 fffffa80`00000006 fffff880`0e49b401 fffffa80`07a84400 : nt!CcMapAndRead+0xc4
fffff880`0e49b310 fffff800`02f0b997 : fffffa80`00030000 fffff880`0e49b450 fffffa80`00001200 fffff880`01820a01 : nt!CcPinFileData+0x570
fffff880`0e49b3d0 fffff800`03117a2a : 00000000`00001200 00000000`05821401 00000000`000001ff fffff880`0e49b528 : nt!CcZeroDataInCache+0x87
fffff880`0e49b480 fffff880`018fd80b : 00000000`00d50e00 fffffa80`0b044bd0 00000000`00008000 fffff880`0e49b780 : nt!CcZeroData+0xee
fffff880`0e49b4e0 fffff880`0182e5c3 : fffffa80`0e52c680 fffff8a0`0eaeb140 fffffa80`07a72380 00000000`00000000 : Ntfs!NtfsZeroData+0xeb
fffff880`0e49b5a0 fffff880`0182ec93 : fffffa80`0e52c680 fffffa80`0b044bd0 fffff880`0e49b701 fffff880`0e49b700 : Ntfs!NtfsCommonWrite+0x3291
fffff880`0e49b750 fffff880`01402bcf : fffffa80`0b044fb8 fffffa80`0b044bd0 fffffa80`0b04b4d0 00000000`00000001 : Ntfs!NtfsFsdWrite+0x1c3
fffff880`0e49b810 fffff880`014016df : fffffa80`0843e8e0 00000000`00000001 fffffa80`0843e800 fffffa80`0b044bd0 : fltmgr!FltpLegacyProcessingAfterPreCallbacksCompleted+0x24f
fffff880`0e49b8a0 fffff800`031cbc3b : 00000000`00000001 fffffa80`07a72380 00000000`00000001 fffffa80`0b044bd0 : fltmgr!FltpDispatch+0xcf
fffff880`0e49b900 fffff800`031d6643 : fffffa80`0b045000 00000000`00000000 fffffa80`07a72380 fffff880`009ea180 : nt!IopSynchronousServiceTail+0xfb
fffff880`0e49b970 fffff800`02ec0e53 : 00000000`00000f01 00000000`00000000 00000000`00000000 00000000`00000000 : nt!NtWriteFile+0x7e2
fffff880`0e49ba70 00000000`74cb2e09 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ fffff880`0e49bae0)
00000000`03e7eda8 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x74cb2e09
Lets take a look at the trap frame.
Code:
0: kd> [COLOR="Green"].trap fffff880`0e49ae70[/COLOR]
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=00000000000c3e33 rbx=0000000000000000 rcx=0000000000000003
rdx=0000000000000001 rsi=0000000000000000 rdi=0000000000000000
rip=[COLOR="red"]fffff80002ef74b7[/COLOR] rsp=fffff8800e49b000 rbp=0000000000000001
r8=0000000000000006 r9=0000000000000000 r10=0000000000000000
r11=0000000000000000 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei pl nz na po nc
nt!KxFlushEntireTb+0xc7:
[COLOR="Red"]fffff800`02ef74b7 [/COLOR]8b05c3e61400 [COLOR="green"]mov[/COLOR] eax,dword ptr [nt!KiTbFlushTimeStamp (fffff800`03045b80)] ds:fffff800`03045b80=000c3e33
Code:
0: kd> [COLOR="green"]knL[/COLOR]
*** Stack trace for last set context - .thread/.cxr resets it
# Child-SP RetAddr Call Site
00 fffff880`0e49b000 fffff800`02ef63cd nt!KxFlushEntireTb+0xc7
01 fffff880`0e49b040 fffff800`02f1a9b0 nt!KeFlushTb+0x119
02 fffff880`0e49b0c0 fffff800`02ed77b4 nt! ?? ::FNODOBFM::`string'+0xada2
03 fffff880`0e49b100 fffff800`02ee6bd4 nt!MmCheckCachedPageStates+0x734
04 fffff880`0e49b2b0 fffff800`02ee6050 nt!CcMapAndRead+0xc4
05 fffff880`0e49b310 fffff800`02f0b997 nt!CcPinFileData+0x570
06 fffff880`0e49b3d0 fffff800`03117a2a nt!CcZeroDataInCache+0x87
07 fffff880`0e49b480 fffff880`018fd80b nt!CcZeroData+0xee
08 fffff880`0e49b4e0 fffff880`0182e5c3 Ntfs!NtfsZeroData+0xeb
09 fffff880`0e49b5a0 fffff880`0182ec93 Ntfs!NtfsCommonWrite+0x3291
0a fffff880`0e49b750 fffff880`01402bcf Ntfs!NtfsFsdWrite+0x1c3
0b fffff880`0e49b810 fffff880`014016df fltmgr!FltpLegacyProcessingAfterPreCallbacksCompleted+0x24f
0c fffff880`0e49b8a0 fffff800`031cbc3b fltmgr!FltpDispatch+0xcf
0d fffff880`0e49b900 fffff800`031d6643 nt!IopSynchronousServiceTail+0xfb
0e fffff880`0e49b970 fffff800`02ec0e53 nt!NtWriteFile+0x7e2
0f fffff880`0e49ba70 00000000`74cb2e09 nt!KiSystemServiceCopyEnd+0x13
10 00000000`03e7eda8 00000000`00000000 0x74cb2e09
This is the stack at the time of the interrupt.
Lets disassemble the instructions.
Code:
0: kd> [COLOR="green"]u @rip[/COLOR]
nt!KxFlushEntireTb+0xc7:
fffff800`02ef74b7 8b05c3e61400 [COLOR="Red"]mov[/COLOR] eax,dword ptr [nt!KiTbFlushTimeStamp (fffff800`03045b80)]
fffff800`02ef74bd a801 test al,1
fffff800`02ef74bf 75e6 [COLOR="red"]jne[/COLOR] nt!KxFlushEntireTb+0xb7 (fffff800`02ef74a7)
fffff800`02ef74c1 f00fba2db6e6140000 lock bts dword ptr [nt!KiTbFlushTimeStamp (fffff800`03045b80)],0
fffff800`02ef74ca 0f834bffffff jae nt!KxFlushEntireTb+0x2b (fffff800`02ef741b)
fffff800`02ef74d0 ebd5 jmp nt!KxFlushEntireTb+0xb7 ([COLOR="green"]fffff800`02ef74a7[/COLOR])
[COLOR="green"]fffff800`02ef74d2[/COLOR] 90 nop
fffff800`02ef74d3 90 nop
Code:
0: kd> [COLOR="Green"]u fffff800`02ef74a7 fffff800`02ef74d2[/COLOR]
nt!KxFlushEntireTb+0xb7:
fffff800`02ef74a7 ffc3 inc ebx
fffff800`02ef74a9 851d191f2000 test dword ptr [nt!HvlLongSpinCountMask (fffff800`030f93c8)],ebx [COLOR="Blue"]<-- testing to see if it's not zero[/COLOR]
fffff800`02ef74af 0f840ee1f5ff je nt! ?? ::FNODOBFM::`string'+0x7400 (fffff800`02e555c3) [COLOR="blue"]<-- jump if not zero[/COLOR]
fffff800`02ef74b5 f390 [COLOR="Red"]pause[/COLOR]
fffff800`02ef74b7 8b05c3e61400 [COLOR="red"]mov[/COLOR] eax,dword ptr [nt!KiTbFlushTimeStamp (fffff800`03045b80)]
fffff800`02ef74bd a801 test al,1
fffff800`02ef74bf 75e6 jne nt!KxFlushEntireTb+0xb7 (fffff800`02ef74a7)
fffff800`02ef74c1 f00fba2db6e6140000 lock bts dword ptr [nt!KiTbFlushTimeStamp (fffff800`03045b80)],0
fffff800`02ef74ca 0f834bffffff jae nt!KxFlushEntireTb+0x2b (fffff800`02ef741b)
fffff800`02ef74d0 ebd5 jmp nt!KxFlushEntireTb+0xb7 (fffff800`02ef74a7)
fffff800`02ef74d2 90 nop
It looks like the thread was executing the pause function at the time of the interrupt in a look waiting for a release.
Well processor #0 was interrupted whilst perform a flush of the Trnaslation Lookaside Buffers which improve translation speed between virtual memory addresses.
Lets take a look at Processor 1s call stack then.
Code:
1: kd> [COLOR="green"]kv[/COLOR]
Child-SP RetAddr : Args to Child : Call Site
fffff880`04a86790 fffff800`02ef63cd : 00000000`00000000 00000000`00000001 fffff6fb`40000170 fffffa80`07a17b50 : nt!KxFlushEntireTb+0xbf
fffff880`04a867d0 fffff800`02f1a9b0 : 00000000`00000001 fffff680`0002e210 00000000`00000001 00000000`00001cd0 : nt!KeFlushTb+0x119
fffff880`04a86850 fffff800`02ed178f : fffff680`0002e210 fffff880`04a86900 fffff700`01080000 fffffa80`0755bec8 : nt! ?? ::FNODOBFM::`string'+0xada2
fffff880`04a86890 fffff800`02ece3fe : 00000000`00000001 00000000`05c42fff fffff880`04a86ae0 fffff680`0002e210 : nt!MiResolveDemandZeroFault+0x1ff
fffff880`04a86980 fffff800`02ebfcee : 00000000`00000001 00000000`05c42fff 00000000`00000001 00000000`000000ec : nt!MmAccessFault+0x5de
fffff880`04a86ae0 00000000`045fbf12 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiPageFault+0x16e (TrapFrame @ [COLOR="Green"]fffff880`04a86ae0[/COLOR])
00000000`002ede04 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x45fbf12
Code:
1: kd> [COLOR="green"].trap fffff880`04a86ae0[/COLOR]
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=0000000005c43000 rbx=0000000000000000 rcx=0000000000000004
rdx=0000000005c42ffc rsi=0000000000000000 rdi=0000000000000000
rip=00000000045fbf12 rsp=00000000002ede04 rbp=00000000002ede0c
r8=000000000000002b r9=000000007737004a r10=0000000000000000
r11=000000000012e2c0 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei pl nz ac po nc
0023:045fbf12 ?? ???
Code:
1: kd> [COLOR="green"]u @rip[/COLOR]
00000000`045fbf12 ?? ???
[COLOR="Red"]^ Memory access error in 'u @rip'[/COLOR]
Can't seem to access the rip register, looks like the Translation Lookaside Buffer flush function is looping.
Code:
2: kd> [COLOR="green"]kv[/COLOR]
Child-SP RetAddr : Args to Child : Call Site
fffff880`0e625790 fffff800`02ef63cd : 00000000`00000000 00000000`00000001 00000000`74cb2450 00000000`0349f170 : nt!KxFlushEntireTb+0xc5
fffff880`0e6257d0 fffff800`02f1a9b0 : 00000000`00000001 fffff680`00061b08 00000000`00000001 00000000`00000003 : nt!KeFlushTb+0x119
fffff880`0e625850 fffff800`02ed178f : fffff680`00061b08 fffff880`0e625900 00000000`00000000 fffffa80`0e708ec8 : nt! ?? ::FNODOBFM::`string'+0xada2
fffff880`0e625890 fffff800`02ece3fe : 00000000`00000001 00000000`0c361000 fffff880`0e625ae0 fffff680`00061b08 : nt!MiResolveDemandZeroFault+0x1ff
fffff880`0e625980 fffff800`02ebfcee : 00000000`00000001 00000000`0c361000 00000000`00000001 00000000`00003131 : nt!MmAccessFault+0x5de
fffff880`0e625ae0 00000000`00571190 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiPageFault+0x16e (TrapFrame @ [COLOR="Green"]fffff880`0e625ae0[/COLOR])
00000000`0359fdf0 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x571190
Same thing with processor #2.
Code:
3: kd> [COLOR="green"]kv[/COLOR]
Child-SP RetAddr : Args to Child : Call Site
fffff880`033ffb58 fffff800`02eca709 : 00000000`001dbc57 fffffa80`0cc9a9b8 00000000`00000000 00000000`00000000 : [COLOR="green"]intelppm!MWaitIdle+0x19[/COLOR]
fffff880`033ffb60 fffff800`02eb989c : fffff880`033d7180 fffff880`00000002 00000000`00000002 fffff800`00000000 : nt!PoIdle+0x52a
fffff880`033ffc40 00000000`00000000 : fffff880`03400000 fffff880`033fa000 fffff880`033ffc00 00000000`00000000 : [COLOR="Green"]nt!KiIdleLoop+0x2c[/COLOR]
Processor #3 is asleep waiting to be woken.
Code:
4: kd> [COLOR="Green"]kv[/COLOR]
Child-SP RetAddr : Args to Child : Call Site
fffff880`20f01680 fffff800`02ef63cd : 00000000`00000000 00000000`00000001 00000000`00001761 fffff800`02ed3fd2 : nt!KxFlushEntireTb+0x93
fffff880`20f016c0 fffff800`02f1a9b0 : 00000000`00000001 fffff680`0016ddc0 00000000`00000001 00000000`00000000 : nt!KeFlushTb+0x119
fffff880`20f01740 fffff800`02ed178f : fffff680`0016ddc0 fffff880`20f017c0 fa800796`9dc80460 fffffa80`08621ec8 : nt! ?? ::FNODOBFM::`string'+0xada2
fffff880`20f01780 fffff800`02edebfe : 00000000`00000001 00000000`2dbb8000 fffff880`20f01ae0 fffff680`0016ddc0 : nt!MiResolveDemandZeroFault+0x1ff
fffff880`20f01870 fffff800`02ece179 : 00000000`00000001 ffffffff`ffffffff fffffa80`0b339b50 fffff680`00171c60 : nt!MiDispatchFault+0x8ce
fffff880`20f01980 fffff800`02ebfcee : 00000000`00000001 00000000`2dbb8000 00000000`26639401 00000000`01af0010 : nt!MmAccessFault+0x359
fffff880`20f01ae0 00000000`0f6c1e89 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiPageFault+0x16e (TrapFrame @ [COLOR="green"]fffff880`20f01ae0[/COLOR])
00000000`0033eca8 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0xf6c1e89
Processor #4 is the same as #1 and #2.
Code:
5: kd> [COLOR="green"]kv[/COLOR]
Child-SP RetAddr : Args to Child : Call Site
fffff880`034cdb58 fffff800`02eca709 : 00000000`001dbc57 fffffa80`08210c68 00000000`00000000 00000000`00000000 : [COLOR="green"]intelppm!MWaitIdle+0x19[/COLOR]
fffff880`034cdb60 fffff800`02eb989c : fffff880`034a5180 fffff880`00000002 00000000`00000002 fffff800`00000000 : nt!PoIdle+0x52a
fffff880`034cdc40 00000000`00000000 : fffff880`034ce000 fffff880`034c8000 fffff880`034cdc00 00000000`00000000 : [COLOR="Green"]nt!KiIdleLoop+0x2c[/COLOR]
Processor #5 is asleep.
Lets take a look at the problematic processor which is #6.
Code:
6: kd> [COLOR="Green"]kv[/COLOR]
Child-SP RetAddr : Args to Child : Call Site
fffff880`0353fb58 fffff800`02eca709 : 00000000`001dbc57 fffffa80`0821a4f8 fffff880`03517180 00000000`00000000 : [COLOR="Red"]intelppm!MWaitIdle+0x19[/COLOR]
fffff880`0353fb60 fffff800`02eb989c : fffff880`03517180 fffff880`00000002 00000000`00000002 fffff880`00000000 : nt!PoIdle+0x52a
fffff880`0353fc40 00000000`00000000 : fffff880`03540000 fffff880`0353a000 fffff880`0353fc00 00000000`00000000 : nt!KiIdleLoop+0x2c
It's sleeping and not responding.
Lets dump the raw stack.
There appears to be a lot of network routines involving your realtek ethernet controller.
Code:
fffff880`0353f9a8 fffff880`054e39a7 [COLOR="Red"]Rt64win7+0x79a7[/COLOR]
fffff880`0353f9b0 00000000`00000003
fffff880`0353f9b8 fffffa80`0c11c6a8
fffff880`0353f9c0 00000000`00000000
fffff880`0353f9c8 00000000`00000003
fffff880`0353f9d0 fffffa80`0c4a1010
fffff880`0353f9d8 fffff880`054e392a [COLOR="red"]Rt64win7+0x792a[/COLOR]
fffff880`0353f9e0 fffffa80`0c11c000
fffff880`0353f9e8 fffff880`054e287c [COLOR="red"]Rt64win7+0x687c[/COLOR]
As stated earlier it's the most up to date.
Lastly processor 7 is also asleep.
Code:
7: kd> [COLOR="green"]kv[/COLOR]
Child-SP RetAddr : Args to Child : Call Site
fffff880`035b1b58 fffff800`02eca709 : 00000000`001dbc57 fffffa80`082134f8 00000000`00000000 00000000`00000000 : [COLOR="Green"]intelppm!MWaitIdle+0x19[/COLOR]
fffff880`035b1b60 fffff800`02eb989c : fffff880`03589180 fffff880`00000002 00000000`00000002 fffff800`00000000 : nt!PoIdle+0x52a
fffff880`035b1c40 00000000`00000000 : fffff880`035b2000 fffff880`035ac000 fffff880`035b1c00 00000000`00000000 : [COLOR="Green"]nt!KiIdleLoop+0x2c[/COLOR]