Recurring BSOD Problem

[ssu]

Im running Windows 7 64 bit, and I get a few Blue screens every day (a variety of different ones). If anyone could help solve this problem it would be great. Attached is my minidump folder

Specs:
AMD Phenom II x4 940 Black Edition
Gigabyte GA-MA790GP-UD4H
Radeon 4870 1GB
OCZ Fatal1ty high performace DDR2 memory 4GB (2x2GB)
Sound Blaster x-Fi titanium pci-e sound card
Seagate Barracuda 7200 RPM 3.0GB/s 16Mb cache OEM

 

[kegobeer]

Have you installed any drivers? Also, fill out your system specs in your profile.

 

[ssu]

Have you installed any drivers? Also, fill out your system specs in your profile.

I have updated my profile. Yes i have installed drivers (the newest ones i beleive)

 

[H2SO4]

Im running Windows 7 64 bit, and I get a few Blue screens every day (a variety of different ones). If anyone could help solve this problem it would be great. Attached is my minidump folder

Specs:
AMD Phenom II x4 940 Black Edition
Gigabyte GA-MA790GP-UD4H
Radeon 4870 1GB
OCZ Fatal1ty high performace DDR2 memory 4GB (2x2GB)
Sound Blaster x-Fi titanium pci-e sound card
Seagate Barracuda 7200 RPM 3.0GB/s 16Mb cache OEM

Short version: there's a high likelihood that your machine is experiencing hardware problems. Overclocking or under-cooling should both be ruled out first, then you'd want to run something like memtest for a long time to try to detect unreliable memory.

==========================

Longer version: A whole bunch of different crashes, all seemingly related to memory and memory management.

091609-18595-01.dmp is perhaps the most telling of the recent ones because it describes an issue called instruction pointer misalignment which is caused by a hardware-level defect in the vast majority of cases. In a nutshell, your machine is executing code which doesn't actually exist - instead of executing instruction 1, followed by instruction 2, followed by inst3, it instead tries to execute 1.5, followed by 2.4, followed by 3.6...

To draw an analogy, film cameras have a winder motor which draws film precisely one-frame-at-a-time while the camera is taking photos. Should the winder mechanism malfunction, the camera may end up exposing the adjacent portions of two separate frames during a singe shot - it has lost track of where each frame begins on the film.

Since that particular minidump is >99% hardware, there's very little point in performing in-depth software analysis of the others. While the hardware is unreliable, all software bets are off.

 

[kegobeer]

Have you used this exact hardware with a different operating system before, and if so, did you have BSODs?

 

[ssu]

Ive only tried the current OS. This computer was built around a month ago, and hasn't been overclocked. Are you saying the hardware is defective, or was it something I did (ive done nothing other than put it together basically)?

 

[chuckr]

091609-18595-01.dmp is perhaps the most telling of the recent ones because it describes an issue called instruction pointer misalignment which is caused by a hardware-level defect in the vast majority of cases.

H2S04,
Just curious:

What provides you the analysis of the 'mini-dump' -- Crash Analyzer in MS Debug?

The IP used to be a CPU register.
Malfunction here would execute various memory locations that actually exist (assuming correct fetchs),
just not in the correct, or intended order.

Any links to the issue called instruction pointer misalignment?
I'm not trying to be a wise-azz here, just trying to learn a little bit more.

I seem to remember a SF member replacing a stock AMD multicore CPU. No o/c, etc.
In and of itself, this seems highly unusual...

Thank you,
Chuck

 

[kegobeer]

Since I don't analyze dump files, I would appreciate if you could post two or three of the BSOD crash screens/error codes.

 

[H2SO4]

The IP used to be a CPU register.
Malfunction here would execute various memory locations that actually exist (assuming correct fetchs),
just not in the correct, or intended order.

Any links to the issue called instruction pointer misalignment?
I'm not trying to be a wise-azz here, just trying to learn a little bit more.

I salute you for being interested in old-school computery stuff

I'll aim to explain the "what" first, then the why...Here's the stack of the crashing thread at the time the user sees blue:

[FONT=Lucida Console][SIZE=1]0: kd> kv[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]Child-SP          RetAddr           : Args to Child                                                           : Call Site[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`031af358 fffff800`02c884e9 : 00000000`0000000a 00000000`00000020 00000000`00000002 00000000`00000001 : nt!KeBugCheckEx[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`031af360 fffff800`02c87160 : 00000000`00000000 00000000`00000004 00000000`00000000 00000000`00000000 : nt!KiBugCheckDispatch+0x69[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`031af4a0 fffff880`014e7f4b : 00000000`00000001 fffff880`01687391 00000000`00000014 fffffa80`00000000 : nt!KiPageFault+0x260 ([COLOR=red]TrapFrame @ fffff880`031af4a0[/COLOR])[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`031af630 00000000`00000001 : fffff880`01687391 00000000`00000014 fffffa80`00000000 00000000`00000014 : [COLOR=blue]ndis+0x1f4b[/COLOR][/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`031af638 fffff880`01687391 : 00000000`00000014 fffffa80`00000000 00000000`00000014 00000000`00000010 : 0x1[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`031af640 00000000`00000014 : fffffa80`00000000 00000000`00000014 00000000`00000010 00000000`00000000 : tcpip+0x7f391[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`031af648 fffffa80`00000000 : 00000000`00000014 00000000`00000010 00000000`00000000 00000000`000007be : 0x14[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`031af650 00000000`00000014 : 00000000`00000010 00000000`00000000 00000000`000007be 00000000`00000078 : 0xfffffa80`00000000[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`031af658 00000000`00000010 : 00000000`00000000 00000000`000007be 00000000`00000078 fffff880`014028b1 : 0x14[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`031af660 00000000`00000000 : 00000000`000007be 00000000`00000078 fffff880`014028b1 fffff880`00000001 : 0x10[/SIZE][/FONT]

And the register context:

[FONT=Lucida Console][SIZE=1]0: kd> r[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]rax=fffff880031af460 rbx=0000000000000004 rcx=000000000000000a[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]rdx=0000000000000020 rsi=0000000000000000 rdi=0000000000000000[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]rip=fffff80002c88f80 rsp=fffff880031af358 rbp=fffff880031af520[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]r8=0000000000000002  r9=0000000000000001 r10=fffff880014e7f4b[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]r11=fffff6fb40000000 r12=0000000000000000 r13=0000000000000000[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]r14=0000000000000000 r15=0000000000000000[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]iopl=0         nv up ei ng nz na po nc[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]cs=0010  ss=0018  ds=002b  es=002b  fs=0053  gs=002b             efl=00000286[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]nt!KeBugCheckEx:[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff800`02c88f80 48894c2408      mov     qword ptr [rsp+8],rcx ss:0018:fffff880`031af360=000000000000000a[/SIZE][/FONT]

But that's not the crashing code - that's the "last resort" mechanism which elevates IRQL to 31 (the highest), thereby interrupting any and all other activity in order to dump physical memory to disk because an irrecoverable error has occurred.

To see the crashing code, we'd have to set the register context as it looked at the time of the fault. Thankfully, the OS dumps a trap frame (in red above) which records that info. To change the context:

[FONT=Lucida Console][SIZE=1]0: kd> .trap fffff880`031af4a0[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]NOTE: The trap frame does not contain all registers.[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]Some register values may be zeroed or incorrect.[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]rax=0000000000000008 rbx=0000000000000000 rcx=fffffa8004861b80[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]rdx=0000000000000001 [COLOR=magenta]rsi=0000000000000000[/COLOR] rdi=0000000000000000[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1][COLOR=red]rip=fffff880014e7f4b[/COLOR] rsp=fffff880031af630 rbp=fffffa8004861a00[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]r8=0000000000000000 r9=0000000000000000 r10=fffffa8004861c00[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]r11=fffff880031af938 r12=0000000000000000 r13=0000000000000000[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]r14=0000000000000000 r15=0000000000000000[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]iopl=0 nv up ei pl nz na pe cy[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]ndis+0x1f4b:[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1][COLOR=red]fffff880`014e7f4b[/COLOR] [COLOR=red]896e20[/COLOR] mov dword ptr [COLOR=magenta][rsi+20h],[/COLOR]ebp ds:5c70:00000000`00000020=????????[/SIZE][/FONT]

OK, so the immediate cause of a crash is a function supposedly at ndis+0x1f4b which tries to move the contents of EBP into the memory address pointed at by RSI plus 0x20 bytes. Trouble is, RSI happens to be zero at the time, so the instruction turns into an attempt to write to memory address 0x20, which is not only in user-mode but also within the first 64KB "guard" block which is always designated no-access in order to catch null pointer references.

Note the (64-bit) RIP instruction pointer though: rip=fffff880014e7f4b

So the offending function which causes the crash (in green) is supposedly at that address, pointed up by RIP. Let's disassemble the code around that address:

[FONT=Lucida Console][SIZE=1]0: kd> u fffff880`014e7f4b[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]ndis+0x1f4b:[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`014e7f[COLOR=red]4b[/COLOR] [COLOR=red]896e20[/COLOR] mov dword ptr [[COLOR=magenta]rsi+20h[/COLOR]],ebp[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`014e7f4e 48897e08 mov qword ptr [rsi+8],rdi[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`014e7f52 48897e10 mov qword ptr [rsi+10h],rdi[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`014e7f56 48897e18 mov qword ptr [rsi+18h],rdi[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`014e7f5a 48897e38 mov qword ptr [rsi+38h],rdi[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`014e7f5e 48897e30 mov qword ptr [rsi+30h],rdi[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`014e7f62 48897e68 mov qword ptr [rsi+68h],rdi[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`014e7f66 48897e60 mov qword ptr [rsi+60h],rdi[/SIZE][/FONT]

For reasons not simple to explain succinctly, that looks a little odd. Let's start the disassembly a bit higher up:

[FONT=Lucida Console][SIZE=1]0: kd> u fffff880`014e7f40[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]ndis+0x1f40:[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`014e7f40 f6 ???[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`014e7f41 0f8413840100 je ndis+0x1a35a (fffff880`0150035a)[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`014e7f47 48893e mov qword ptr [rsi],rdi[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`014e7f4a 48896e20 mov qword ptr [rsi+20h],rbp[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`014e7f4e 48897e08 mov qword ptr [rsi+8],rdi[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`014e7f52 48897e10 mov qword ptr [rsi+10h],rdi[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`014e7f56 48897e18 mov qword ptr [rsi+18h],rdi[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`014e7f5a 48897e38 mov qword ptr [rsi+38h],rdi[/SIZE][/FONT]

Hold on, that looks different, although it's still not right because I happen to have arbitrarily started disassembly in the middle of an instruction. Let's go back a little more:

[FONT=Lucida Console][SIZE=1]0: kd> u fffff880`014e7f3a[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]ndis+0x1f3a:[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`014e7f3a 8b6c2430 mov ebp,dword ptr [rsp+30h][/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`014e7f3e 4885f6 test rsi,rsi[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`014e7f41 0f8413840100 je ndis+0x1a35a (fffff880`0150035a)[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`014e7f47 48893e mov qword ptr [rsi],rdi[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`014e7f[COLOR=seagreen]4a[/COLOR] [COLOR=blue]48[/COLOR][COLOR=red]896e20[/COLOR] mov qword ptr [rsi+20h],rbp[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`014e7f[COLOR=seagreen]4e[/COLOR] 48897e08 mov qword ptr [rsi+8],rdi[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`014e7f52 48897e10 mov qword ptr [rsi+10h],rdi[/SIZE][/FONT]
[FONT=Lucida Console][SIZE=1]fffff880`014e7f56 48897e18 mov qword ptr [rsi+18h],rdi[/SIZE][/FONT]

That looks like real code. There's a "test" followed by a conditional jump (je), and then we're moving a bunch of structured info into memory starting with [RSI].

Note that there is no instruction at 0xfffff880`014e7f4b! There are (legit) instructions starting at '4a and '4e, but the attempt to execute an instruction starting at '4b (in red) leads to an inevitable mangling of the instruction itself.

Instead of this:
fffff880`014e7f4a 48896e20 mov qword ptr [rsi+20h],rbp


...RIP is one byte too far along and we're executing an instruction which doesn't exist:

fffff880`014e7f4b 896e20 mov dword ptr [rsi+20h],ebp

And as for the "why"... (to be continued)

 

[H2SO4]

(continuation from previous lengthy post)

Remember that on IA-32 and AMD64/EM64T architectures the code doesn't directly set the RIP. Instead, the hardware itself has the job of scanning through the memory which is the instruction sequence and determining where one instruction ends and the next one begins ("non-orthogonal" instruction set). Every time it concludes that it has isolated the next instruction and set in motion its execution, RIP is automatically updated without any involvement from software. In other words, it is almost impossible for software bugs to cause RIP to point in between two instructions because it's the hardware which moves RIP around.

One class of software bug which can lead to RIP getting whacked is when an errant memory write mangles the contents of a thread's CONTEXT structure - the one which contains the values of its registers, especially while the thread is parked and waiting for processor time. For example, if my buggy driver happens to accidentally throw a wild write to location XYZ, and there happens to be a CONTEXT structure stored thereabouts, then the victim thread could have one or more of its register contents - including RIP - point to something totally invalid the next time it gets scheduled for proc time.

That is exceptionally rare though, and when it does happen the results are almost always stupidly inaccurate, instead of being off-by-one byte as in this case. In situations like this one where RIP is only off by one or a few bytes, the cause is almost certainly unreliable hardware. A transistor farted somewhere

In case you should think I go through this rigmarole for every minidump, let me assure you all I did was to double-click the file. Because I've got the debugger set up quite nicely, it opens the dump, tries to have an automated go at deducing the cause of any crash conditions it finds in there, and spits out a text result thus:

SYMBOL_NAME: ndis+1f4b
FOLLOWUP_NAME: MachineOwner
IMAGE_NAME: hardware
DEBUG_FLR_IMAGE_TIMESTAMP: 0
MODULE_NAME: hardware
FAILURE_BUCKET_ID: X64_IP_MISALIGNED

I wrote up a short intro on how to get WinDBG to do the heavy lifting over on vistax64.com:

http://www.vistax64.com/tutorials/221510-crash-dumps-analyse-bugcheck-process.html

I enjoy this stuff so if there's something else you wanted to discuss - let's talk

 

[H2SO4]

Ive only tried the current OS. This computer was built around a month ago, and hasn't been overclocked. Are you saying the hardware is defective, or was it something I did (ive done nothing other than put it together basically)?

I'm 99% confident that your hardware exhibited a defect in 091609-18595-01.dmp. It is virtually impossible to tell (from a minidump) whether the cause is absolutely _broken_ hardware or something like insufficient cooling or a speck of dust lodged in a connector.

Personally, I'd start by running a memory tester for days if necessary - long enough to either find something or convince myself that I'm not going to find unreliable memory using that method.

 

[chuckr]

Holy Schmoley!
Real 'computer stuff'!!!

Thanks a million! Your Vista tut looks like a good start point.

DOS Debug in new clothes... (U = Unassemble, etc...)

You got a decent (free) standalone Disassembler to recommend?
Found IDA pro freebie, but ain't looked hard at it yet.

Just wanted to throw a quick reply, to say thanks!
Ditto the extended Memtest, but no CPU exercising there.
MS or Intel or AMD got any hard bangers that you know of?

Ooops, disregard wait -- thought I saw something, but mistaken..

 

[chuckr]

fffff880`014e7f4a 48896e20 mov qword ptr [rsi+20h],rbp
fffff880`014e7f4e 48897e08 mov qword ptr [rsi+8],rdi
fffff880`014e7f52 48897e10 mov qword ptr [rsi+10h],rdi
fffff880`014e7f56 48897e18 mov qword ptr [rsi+18h],rdi

The IP seems to count OK:
No instruction at 4b may be because of correct IP increment,
(+1 per byte, viz):
4a = 48 89 6e 20
4b is the '89',
4c is the '6e',
4d is the '20',

then,
4e 48 89 7e 08 <---< is appropriate, count-wise.

What strikes me (as quick first look) is the "Increment" of the
"LOOP" or "REPx" instruction, not proceeding logically:
Starting at +00h, then bumping up by +08h.....

Code:

0: kd> u fffff880`014e7f3a
ndis+0x1f3a:
fffff880`014e7f3a 8b6c2430 mov ebp,dword ptr [rsp+30h]
fffff880`014e7f3e 4885f6 test rsi,rsi
fffff880`014e7f41 0f8413840100 je ndis+0x1a35a (fffff880`0150035a)
fffff880`014e7f47 48893e mov qword ptr [rsi],rdi
fffff880`014e7f4a 48896e20 mov qword ptr [rsi+20h],rbp <===< *****
fffff880`014e7f4e 48897e08 mov qword ptr [rsi+8],rdi
fffff880`014e7f52 48897e10 mov qword ptr [rsi+10h],rdi
fffff880`014e7f56 48897e18 mov qword ptr [rsi+18h],rdi

The problem seems (to me) that this instruction shud be:

48897e00 mov qword ptr [rsi+00h],rdi

Perhaps something wrong, not in IP, but the 32-bit word itself...
===============================================================================

Isolating:
---------
fffff880`014e7f4a 48 89 6e 20 mov qword ptr [rsi+20h],rbp <===< *****
fffff880`014e7f4a 48 89 7e 00 mov qword ptr [rsi+00h],rdi

Expanding more:
--------------
___6 ___e ___2 ___0 hex
0110 1110 0010 0000 bin

___7 ___e ___0 ___0 hex
0111 1110 0000 0000 bin
___^______ ^_____ <--- Specifically these bits: (2 to the 5th), (2 to the 12th)


Temporarily disregarding, of course:

"IA processors are “little endian” machines..."

 

[H2SO4]

fffff880`014e7f4a 48896e20 mov qword ptr [rsi+20h],rbp
fffff880`014e7f4e 48897e08 mov qword ptr [rsi+8],rdi
fffff880`014e7f52 48897e10 mov qword ptr [rsi+10h],rdi
fffff880`014e7f56 48897e18 mov qword ptr [rsi+18h],rdi

The IP seems to count OK:
No instruction at 4b may be because of correct IP increment,
(+1 per byte, viz):
4a = 48 89 6e 20
4b is the '89',
4c is the '6e',
4d is the '20',

then,
4e 48 89 7e 08 <---< is appropriate, count-wise.

What strikes me (as quick first look) is the "Increment" of the
"LOOP" or "REPx" instruction, not proceeding logically:
Starting at +00h, then bumping up by +08h.....

Code:

0: kd> u fffff880`014e7f3a
ndis+0x1f3a:
fffff880`014e7f3a 8b6c2430 mov ebp,dword ptr [rsp+30h]
fffff880`014e7f3e 4885f6 test rsi,rsi
fffff880`014e7f41 0f8413840100 je ndis+0x1a35a (fffff880`0150035a)
fffff880`014e7f47 48893e mov qword ptr [rsi],rdi
fffff880`014e7f4a 48896e20 mov qword ptr [rsi+20h],rbp <===< *****
fffff880`014e7f4e 48897e08 mov qword ptr [rsi+8],rdi
fffff880`014e7f52 48897e10 mov qword ptr [rsi+10h],rdi
fffff880`014e7f56 48897e18 mov qword ptr [rsi+18h],rdi

The problem seems (to me) that this instruction shud be:

48897e00 mov qword ptr [rsi+00h],rdi

Perhaps something wrong, not in IP, but the 32-bit word itself...

Yep, just like DOS's "debug". Have rep

IDA has amazing graphical disassembly capabilities which are immensely useful when trying to understand other people's code - especially without access to "private" symbols. Between WinDBG/CDB/NTSD (all based on the same engine) and IDA, I haven't had any need to use another dedicated disassembler for a long time.

With regards to memtest and CPU stress test utils, I'm not well versed in those techniques. To me, the software's the interesting bit, and I'll happily defer to those with more knowledge when it comes to figuring out whether it's the third DIMM from the left or the APIC that's responsible for any given hardware weirdness.

As far as the IP is concerned, let's go a lot further back and start disassembly from there:

0: kd> u fffff880`014e7f10 L12
ndis+0x1f10:
fffff880`014e7f10 0041ff add byte ptr [rcx-1],al
fffff880`014e7f13 842c94 test byte ptr [rsp+rdx*4],ch
fffff880`014e7f16 0100 add dword ptr [rax],eax
fffff880`014e7f18 00ff add bh,bh
fffff880`014e7f1a 15b9310500 adc eax,531B9h
fffff880`014e7f1f 488bf0 mov rsi,rax
fffff880`014e7f22 4885c0 test rax,rax
fffff880`014e7f25 0f84aa010000 je ndis+0x20d5 (fffff880`014e80d5)
fffff880`014e7f2b 4885f6 test rsi,rsi
fffff880`014e7f2e 0f843a020000 je ndis+0x216e (fffff880`014e816e)
fffff880`014e7f34 448b642450 mov r12d,dword ptr [rsp+50h]
fffff880`014e7f39 4c8b6c2430 mov r13,qword ptr [rsp+30h]
fffff880`014e7f3e 4885f6 test rsi,rsi
fffff880`014e7f41 0f8413840100 je ndis+0x1a35a (fffff880`0150035a)
fffff880`014e7f47 48893e mov qword ptr [rsi],rdi
fffff880`014e7f[COLOR=royalblue]4a[/COLOR] 48896e20 [COLOR=seagreen]mov qword ptr [rsi+20h],rbp[/COLOR]
fffff880`014e7f[COLOR=royalblue]4e[/COLOR] 48897e08 mov qword ptr [rsi+8],rdi
fffff880`014e7f52 48897e10 mov qword ptr [rsi+10h],rdi

Going that far back, we can be almost certain that the disassembly has correctly "snapped to" by the time it reaches '4a. According to the in-memory version of the ndis.sys image, the instruction at '4a is 4 bytes long ("48896e20").

The problem with the IP is that it _should_ be at '4a or '4e, but certainly not '4b, because even if the in-memory NDIS image does not match the on-disk binary, the disassembly above shows the code that the machine ought to have been following.

The problem seems (to me) that this instruction shud be:

48897e00 mov qword ptr [rsi+00h],rdi

Perhaps something wrong, not in IP, but the 32-bit word itself...

Quite possible. The in-memory image may be corrupted. We'd need somebody with a 7100 build (like the OP) to verify what's meant to be at that location. Bit-flips due to hardware issues may be responsible for both the IP misalignment and corruption of the binary image.

By itself, corruption of the in-memory image won't cause the IP to be misaligned though. However badly misrepresented the code may be in memory, perhaps due to software bugs, it's still meant to be precisely followed by the hardware, and the (R)IP should never be between two instructions.

 

[chuckr]

The problem with the IP is that it _should_ be at '4a or '4e, but certainly not '4b, because even if the in-memory NDIS image does not match the on-disk binary, the disassembly above shows the code that the machine ought to have been following.

I believe the caveat here is:
Code:

0: kd> .trap fffff880`031af4a0
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=0000000000000008 rbx=0000000000000000 rcx=fffffa8004861b80
rdx=0000000000000001 rsi=0000000000000000 rdi=0000000000000000
rip=fffff880014e7f4b rsp=fffff880031af630 rbp=fffffa8004861a00
r8=0000000000000000 r9=0000000000000000 r10=fffffa8004861c00
r11=fffff880031af938 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei pl nz na pe cy
ndis+0x1f4b:
fffff880`014e7f4b 896e20 mov dword ptr [rsi+20h],ebp ds:5c70:00000000`00000020=????????

Not knowing exactly how "real-time" the "trap-frame" operates, I would venture to guess that the possibility exists that:
the CPU 'instruction-decoder' was in the midst of updating the IP register
(which may only increment +1 at a time, depending upon hw logic gates),
other factors, other reasonings, etc. ("non-orthogonal" instruction set)

Note the instruction mnemonics:

test rsi,rsi
je elsewhere >--> Heres where I "don't know" what TEST does, to satisfy the "GOTO"

mov [rsi],rdi ; 'Zero' start/dest indices seem correct, so reg_dump agrees

If I were a professional, I'd 'Instruction_Breakpoint' at 7f3e, then poke around the registers (during real-time execution).
Then 'Single-step' and poke around some more.
Maybe setting a 'Data_Breakpoint' or two, at curious areas.

By itself, corruption of the in-memory image won't cause the IP to be misaligned though. However badly misrepresented the code may be in memory, perhaps due to software bugs, it's still meant to be precisely followed by the hardware, and the (R)IP should never be between two instructions.

That's where I would think differently:
the 'non-orthogonal' instruction set and the 'instruction-decoder' could well change the IP register with screwy counts:
Immediately to mind would be this fake example:
Opcode 0x40 = Load Accumulator with register C (maybe 2 bytes, immed)
Opcode 0x41 = Load Acc from Memory location 0x1cad (6 bytes?)
A one-bit change, interpreted by the 'instruction-decoder', would delta the IP by +/- 4 more/less than intended.

Debug of the (good) .exe may not show this on the 2nd load, because only the 1st load (real-time run) exhibited the error.

Surely, there exists a hw problem.... If I were a betting man, the CPU!

Isolation requires in-person 'hand-holding', observation, and trusty 'tools'...

Thanks for the discussion - Gotta fire up the barbie now...

Chuck

Hope we don't both get the BanHammer, for getting this thread:

 

[H2SO4]

Endian little hate I

Couldn't help myself - I had to hunt down a 7100 machine like the OPs and check that range. The NDIS.SYS in-memory image is intact in their case, at least in that location:

Mine:
.text:0000000000011F40 F6 0F 84 13 84 01 00 48 89 3E 48 89 6E 20 48 89 ÷¤ää
.Hë>Hën Hë

OPs:
0: kd> db fffff880`014e7f40 L10
fffff880`014e7f40 f6 0f 84 13 84 01 00 48-89 3e 48 89 6e 20 48 89 .......H.>H.n H.

I used WinDBG for the OPs and IDA for mine, just to confirm. Unsurprisingly, they agree on the disassembly. Perhaps the reference to RSI+20h and RBP is located there for pipelining reasons - some sort of structural or data hazard avoidance.

 

[chuckr]

Endian little hate I

Couldn't help myself - I had to hunt down a 7100 machine like the OPs and check that range.

t_em
!.oo

But I'm gonna hit the 'Off' switch and read your post at Coffee-time...

You really don't know what the OP has...
You're assuming a valid, operable 7100, like you have...

Ta.

 

[H2SO4]

Thanks for a very interesting discussion

You really don't know what the OP has...
You're assuming a valid, operable 7100, like you have...

The aim was to check whether their in-memory version of the instruction at '4a matches ndis.sys on another 7100 box, or whether it was possibly mangled as you suspected (and I agree there was good reason to suspect that). Their minidump tells me some details regarding the OS version labelling and the specifics of their NDIS.SYS binary ('vertarget', '!lmi ndis.sys'), and the comparison to my own (presumed good) copy tells me that there is no discrepancy, at least in that area - the '4a instruction really is "mov [rsi+20h], rbp".

I believe the caveat here is:

Quote:
0: kd> .trap fffff880`031af4a0
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.

The Windows trap frame struct ('dt _KTRAP_FRAME') doesn't necessarily get populated with the contents of every register - only those which are necessary for the operation of a particular type of trap handler: nt!KiTrap0E, nt!KiTrap0A... Hence, the warning above is spewed by the debugger as a generic reminder that at least some of the values may not be accurate. (R)IP is obviously essential though - so we can get back to where we left off when the handler's done - and it'll absolutely be saved every time. In the OPs case, the RIP value is undeniably populated and pointing close to where it should, albeit off-by-one byte:

0: kd> dt _KTRAP_FRAME fffff880`031af4a0 RIP
nt!_KTRAP_FRAME
+0x168 Rip : 0xfffff880`014e7f4b

(That's where the debugger gets its own 'kv' trap info of course.)

Not knowing exactly how "real-time" the "trap-frame" operates, I would venture to guess that the possibility exists that:
the CPU 'instruction-decoder' was in the midst of updating the IP register
(which may only increment +1 at a time, depending upon hw logic gates),

For what it's worth, I don't believe it works that way. From a software point of view, IP updates are an atomic operation, and the OS will never be able to watch the IP "count" incrementally between two valid instruction offsets.

If I were a professional, I'd 'Instruction_Breakpoint' at 7f3e, then poke around the registers (during real-time execution).
Then 'Single-step' and poke around some more.
Maybe setting a 'Data_Breakpoint' or two, at curious areas.

Without wishing to be seen to be buttering up a forum heavyweight, it's very obvious that you're very professional

I agree completely that if we had a setup with a reproducible fault and a kernel debugger hooked up it would be very interesting to 'bp <\'7f3e>', then 't' for a bit, watch if it's going to branch or not in that conditional jump, and 'ba <address>' (hardware breakpoint) on some of the underlying structures. I rather doubt this is reproducible though - if it's hardware, it may never happen again in the same way and in the same place.

That's where I would think differently:
the 'non-orthogonal' instruction set and the 'instruction-decoder' could well change the IP register with screwy counts:

Oh sure, software can directly walk the IP all over the place (the debugger does it all the time), but the point is that the code in question does nothing to directly shunt the IP one byte further along than it should be. Perhaps the thread was pre-empted and its context record was damaged, but the chances are very small that the stored RIP value merely got INCed (or similar).

Surely, there exists a hw problem.... If I were a betting man, the CPU!

Isolation requires in-person 'hand-holding', observation, and trusty 'tools'...

Yes, I certainly wouldn't be putting my money on a software flaw in this particular crash. I don't know whether it's the processor though.

Hope we don't both get the BanHammer, for getting this thread:

I'm prepared to run that risk for the joy I get from discussing these issues

 

[chuckr]

H2S04 1's and 0's

http://www.sevenforums.com/general-discussion/25276-recurring-bsod-problem.html

Didn't want this thing to 'die on the vine', especially since you went through the trouble of verifying the binary file 'ndis.sys'.

Obviously, the OP has the correct code with his ndis.sys and the debug dump also indicates that.

My aiming at the instruction: mov qword ptr [rsi+20h],rbp
was definitely with blinders on, assuming the +08h should have been in an orderly 'progression', starting with +00h.

The bloody +00h is, in fact there, simply not indicated as such.
The TEST and JE mnemonics I've kissed off as checking for 'in bounds' exceptions.

0: kd> u fffff880`014e7f10 L12
ndis+0x1f10:

014e7f10 0041ff add byte ptr [rcx-1],al
014e7f13 842c94 test byte ptr [rsp+rdx*4],ch
014e7f16 0100 add dword ptr [rax],eax
014e7f18 00ff add bh,bh
014e7f1a 15 b9310500 adc eax,531B9h
014e7f1f 488b f0________ mov rsi,rax

014e7f22 4885 c0________ test rax,rax
014e7f25 0f84 aa010000__ je ndis+0x20d5 (014e80d5)

014e7f2b 4885 f6________ test rsi,rsi
014e7f2e 0f84 3a020000 je ndis+0x216e (014e816e)

014e7f34 448b642450_____ mov r12d,dword ptr [rsp+50h]
014e7f39 4c8b6c2430_____ mov r13,qword ptr [rsp+30h]

014e7f3e 4885 f6________ test rsi,rsi
014e7f41 0f84 13840100__ je ndis+0x1a35a (0150035a)

014e7f47 4889 3e________ mov qword ptr [rsi],rdi <===< mov qword ptr [rsi+00h],rdi
014e7f4a 4889 6e 20_____ mov qword ptr [rsi+20h],rbp
014e7f4e 4889 7e 08_____ mov qword ptr [rsi+8],rdi
014e7f52 4889 7e 10_____ mov qword ptr [rsi+10h],rdi

These are excerpts from the Intel IA-32 manual, but might be similar for x64:

TEST — Logical Compare

Description:
This instruction computes the bit-wise logical AND of first operand (source 1 operand) and the
second operand (source 2 operand) and sets the SF, ZF, and PF status flags according to the
result. The result is then discarded.

Operation: (in Pseudo c code)
TEMP ¬ SRC1 AND SRC2;
SF ¬ MSB(TEMP);
IF TEMP = 0
THEN ZF ¬ 1;
ELSE ZF ¬ 0;
FI:
PF ¬ BitwiseXNOR(TEMP[0:7]);
CF ¬ 0;
OF ¬ 0;
(*AF is Undefined*)

Flags Affected:
The OF and CF flags are cleared to 0.
The SF, ZF, and PF flags are set according to the result
(refer to the “Operation” section above). The state of the AF flag is undefined.

Protected Mode Exceptions:
#GP(0) If a memory operand effective address is outside the CS, DS, ES, FS, or
GS segment limit.
If the DS, ES, FS, or GS register contains a null segment selector.
#SS(0) If a memory operand effective address is outside the SS segment limit.
#PF(fault-code) If a page fault occurs.
#AC(0) If alignment checking is enabled and an unaligned memory reference is
made while the current privilege level is 3.

Real-Address Mode Exceptions:
#GP If a memory operand effective address is outside the CS, DS, ES, FS, or
GS segment limit.
#SS If a memory operand effective address is outside the SS segment limit.

Virtual-8086 Mode Exceptions:
#GP(0) If a memory operand effective address is outside the CS, DS, ES, FS, or
GS segment limit.
#SS(0) If a memory operand effective address is outside the SS segment limit.
#PF(fault-code) If a page fault occurs.
#AC(0) If alignment checking is enabled and an unaligned memory reference is
made.
----------------------------------------------------
Jcc — Jump if Condition Is Met (Continued)

Opcode Instruction Description

0F 84 cw/cd JE rel16/32 Jump near if equal (ZF=1)

This instruction checks the state of one or more of the status flags in the EFLAGS register (CF,
OF, PF, SF, and ZF) and, if the flags are in the specified state (condition), performs a jump to the
target instruction specified by the destination operand. A condition code (cc) is associated with
each instruction to indicate the condition being tested for. If the condition is not satisfied, the
jump is not performed and execution continues with the instruction following the Jcc instruction.

Is it 'normal' for 64-bit code to be using Dword pointers?

Does your 7100 with that ndis.sys operate correctly?

Since you like this stuff, is there any way that you could execute this "chunk" of code under 'controlled' circumstances?

 

[H2SO4]

Didn't want this thing to 'die on the vine', especially since you went through the trouble of verifying the binary file 'ndis.sys'. Obviously, the OP has the correct code with his ndis.sys and the debug dump also indicates that.

Glad to talk about it any time

In reality I only verified the contents of the QWord starting at 0xfffff880`014e7f40 - the one that contains the '4a and '4e instructions, as well as the (non-existent) '4b instruction the box seemingly tried to execute at the time of the crash. I cannot be certain whether the OPs entire NDIS.SYS image is correct, mostly because it's not all recorded in the 270KB worth of minidump! (If we had a full kernel dump from the OP we could use the debugger's !chkimg extension to verify the whole thing against a known-good copy.)

My aiming at the instruction: mov qword ptr [rsi+20h],rbp
was definitely with blinders on, assuming the +08h should have been in an orderly 'progression', starting with +00h. The bloody +00h is, in fact there, simply not indicated as such.

Sure, "[rsi]" is equivalent to "[rsi+00h]", if that's what you mean. That's just the way WinDBG chooses to present the disassembled mnemonics.

Is it 'normal' for 64-bit code to be using Dword pointers?

I don't think I understand where you're seeing the 32-bit pointer(s)? They certainly shouldn't be any absolute 32-bit pointers because a jump to such an address would easily take us out of the kernel-mode address space on an x64/IA-64 machine and thereby cause a crash.

There's nothing wrong with 32-bit relative displacement though. Perhaps you're referring to this?

...
fffff880`014e7f3e 4885f6 test rsi,rsi
fffff880`014e7f41 0f8413840100 je ndis+0x1a35a (fffff880`0150035a)
fffff880`014e7f47 48893e mov qword ptr [rsi],rdi
...

That conditional jump at '41 uses a relative offset - relative to the current position. As per the Intel doc you quoted, the 'je' instruction includes a relative ("rel") 16/32-bit pointer:

0F 84 cw/cd JE rel16/32 Jump near if equal (ZF=1).

In the OP's case, the entire instruction is "0f8413840100". Hence, the leading "0f84" is the opcode and "13840100" is the relative offset in little-endian, so 0x18413 to me and you.

The starting point for that displacement is the IP as it would have looked after the jump instruction is fetched (note the atomic nature of the IP update):

fffff880`014e7f41 0f8413840100 je ndis+0x1a35a (fffff880`0150035a)
fffff880`014e7f47 48893e mov qword ptr [rsi],rdi

Adding the 'je' instruction's relative displacement to the IP yields:

0: kd> ? fffff880`014e7f47 + 0x18413
Evaluate expression: -8246315187366 = fffff880`0150035a

Let's now get the load address of NDIS.SYS as a starting point...

0: kd> lmm ndis
start end module name
fffff880`014e6000 fffff880`015d7000 ndis T (no symbols)

If we compare the displacement between the jump's destination and the NDIS load address:

0: kd> ? fffff880`0150035a - fffff880`014e6000
Evaluate expression: 107354 = 00000000`0001a35a

So the final jump destination would be NDIS+0x1a35a, as per WinDBG's calculation highlighted in pink up higher above. To me, it all seems to check out.

Does your 7100 with that ndis.sys operate correctly?

Yes, to the best of my knowledge. It certainly doesn't BSOD.

Since you like this stuff, is there any way that you could execute this "chunk" of code under 'controlled' circumstances?

Sure, no problem. I can run it under a kernel debugger and thus control its operation. If you like, you can drive. Let me know what breakpoints you'd like set and I'll paste back the results. Good geeky fun