New
#31
Are you getting the dumps to load and the !analyze –v opens the basic info?
hey Dave. Yeah, I've been checking a few dmps. I actually found a 0x124 which I believe is hardware, and it actually listed a process name too.....
Its quite intimidating this stuff. I realise I can start with analyse -v, but its all the stuff after that, and it seems you need a fairly good grasp of all the windbg commands, and know exactly what to do next.
I don't know.....this is going to take a damn long time to get to grips with
For some particular bugchecks, there are special techniques to follow ....
STOP 0x101: CLOCK_WATCHDOG_TIMEOUT troubleshtg
STOP 0x101: CLOCK_WATCHDOG_TIMEOUT troubleshtg
STOP 0x116: VIDEO_TDR_ERROR troubleshooting
Stop 0x124 - what it means and what to try
In those cases, follow those links for troubleshooting.
Not damn long ... most probably a couple of week I guess !
The section Arc linked has a lot of good info.
A good source for Stop code info and driver info is usasma's site:
THE BSOD LISTING
Driver Reference Table
Also searching the System and APPs Event Logs can give good clues, use 'Find' and enter 'error', this will get you through it faster.
And don't forget your friend, preferred search engine.
You'll be flying through dumps by the end of next week.
Let us know when you have any questions.
The article on 0x124 crashdumps ARC referred too is great, but is kinda outdated and uses an older, insufficient and more difficult method for gaining info on the problem. Your best bet is to just use the !errrec (yes, that's 3 'r's, not 2, as in error record) 'command'* in Windbg and give it Arg2 of the bugcheck code (2nd of the 4 numbers in parentheses next to the 0x124 number), so it'll be !errrec <Arg2> (no <> of course). That'll give you plenty of info, with certain lines like the actual error message which is a lot easier to read (and reliable) than whipping out an Intel/AMD dev manual and trying to interpret MCi status codes.
Articles that explain further are here and here.
* - In Windbg terminology, a command is something built into the Windbg (KD/CDB) engine, and is usually prefixed with a period (.) like .reload or .process, whereas an extension is something that adds more robust functionality and can be either included with Windbg, made personally or downloaded from a site (like cmkd), and they are prefixed with a bang (!), like !analyze or !thread. I just called !errrec - which is an extension - a command to prevent confusion, but it's really technically called an extension.
Thanks for the information.
The more the better, we are all trying to learn and appreciate your input.
This is like pulling teeth! I'm going to try Frederik's video's over the next few weeks and see if I can find the will to live through this stuff. The basic problem is I'm missing the fundamentals : I have no idea in which order I should proceed. I've grasped that Arg4 is important, but I don't know how to "tease" information from it (decipher the hexdecimal to mean something to me), or even if its the right information. For example:
Code:2: kd> !analyze -v ******************************************************************************* * * * Bugcheck Analysis * * * ******************************************************************************* MEMORY_MANAGEMENT (1a) # Any other values for parameter 1 must be individually examined. Arguments: Arg1: 0000000000041289, The subtype of the bugcheck. Arg2: 000007feee270001 Arg3: 0000000000000b68 Arg4: 000007fcee270005 Debugging Details: ------------------ BUGCHECK_STR: 0x1a_41289 CUSTOMER_CRASH_COUNT: 1 DEFAULT_BUCKET_ID: VISTA_DRIVER_FAULT PROCESS_NAME: TrustedInstallThere is such a vast array of commands available, I wouldn't even know where to start......Code:2: kd> lmvm nt* start end module name fffff800`03002000 fffff800`035ea000 nt (pdb symbols) c:\symcache\ntkrnlmp.pdb\B2DA40502FA744C18B9022FD187ADB592\ntkrnlmp.pdb Loaded symbol image file: ntkrnlmp.exe Mapped memory image file: c:\symcache\ntoskrnl.exe\503F82BE5e8000\ntoskrnl.exe Image path: ntkrnlmp.exe Image name: ntkrnlmp.exe Timestamp: Fri Aug 31 00:41:58 2012 (503F82BE) CheckSum: 00554126 ImageSize: 005E8000 File version: 6.1.7601.17944 Product version: 6.1.7601.17944 File flags: 0 (Mask 3F) File OS: 40004 NT Win32 File type: 1.0 App File date: 00000000.00000000 Translations: 0409.04b0 CompanyName: Microsoft Corporation ProductName: Microsoft® Windows® Operating System InternalName: ntkrnlmp.exe OriginalFilename: ntkrnlmp.exe ProductVersion: 6.1.7601.17944 FileVersion: 6.1.7601.17944 (win7sp1_gdr.120830-0333) FileDescription: NT Kernel & System LegalCopyright: © Microsoft Corporation. All rights reserved. fffff880`01235000 fffff880`013d8000 Ntfs (deferred) Mapped memory image file: c:\symcache\Ntfs.sys\5040D4C61a3000\Ntfs.sys Image path: \SystemRoot\System32\Drivers\Ntfs.sys Image name: Ntfs.sys Timestamp: Sat Sep 01 00:44:14 2012 (5040D4C6) CheckSum: 0019EA7B ImageSize: 001A3000 File version: 6.1.7601.17945 Product version: 6.1.7601.17945 File flags: 0 (Mask 3F) File OS: 40004 NT Win32 File type: 3.7 Driver File date: 00000000.00000000 Translations: 0409.04b0 CompanyName: Microsoft Corporation ProductName: Microsoft® Windows® Operating System InternalName: ntfs.sys OriginalFilename: ntfs.sys ProductVersion: 6.1.7601.17945 FileVersion: 6.1.7601.17945 (win7sp1_gdr.120831-0331) FileDescription: NT File System Driver LegalCopyright: © Microsoft Corporation. All rights reserved.
There's just some types of crashdumps that are indecipherable because they actually provide details only accessible to those with private access to Windows code, like Windows development and support teams. The 0x1A crashes are often a prime example. The only item of much importance with a 0x1A bugcheck is Arg1, which is the subcode. If the subcode present is not mentioned in the list offered by the Windbg help manual for that particular bugcheck, then just disregard it and consider the crashdump worthless.
Even the best and brightest kernel debugging analysts can be easily stumped by a crashdump. Not every crashdump provided can offer relevant information. Some of them just happen too late and the culprit has already left the scene, or they just offer too little detail. The best step for a person doing debugging of this nature is to first make it so that a crashdump will provide something meaningful. Often this would mean turning on Driver Verifier, doing manual crashing, reading a kernel dump instead of a minidump, collecting more minidumps, etc. etc.. If what you have right now is making you scratch your head, STOP, and get more information. Driver Verifier is always the best start for this.
Yes, there's a lot of commands, and there's a lot of information, because we're dealing with a robust OS from the ground up, so learning about all of its intricate parts can kinda be necessary. I have recommendations on literature, sites and whatnot for research purposes, as well as studying tips (like running through Windbg help manual on each command) here, but it's gonna boil down to how much you're willing to learn on it.
Kernel debugging and crashdump analysis is not an entry-level profession. The people usually responsible for dealing with it are escalation engineers which are the highest echelon of computer engineering in the industry. However, it is also not an insurmountable obstacle. There is plenty of documentation and resources available to make the trek easier, but it's going to rely a lot on exposure and experience, which fortunately for here at SF is available in droves. Don't swallow more than you can stomach, and don't bite more than you can chew, and you should be able to make progress in understanding more and more about this field of study.
It's slow and methodical, but the benefits are great. Computer technicians and sysadmins that are able to do forensic troubleshooting like this are very, very few and far inbetween, and makes them stand head and shoulders over the rest of their peers in the industry. Knowledge like this will certainly not go unheeded, as it is a rare and treasured gem which few know of its value until they see it firsthand.
Thats great feedback! Thanks. Yes, you summed it up nicely : I'm certainly at an entry-level trying to see what I can accomplish with the hardcore end of the business. Its frustrating to say the least, primarily due to my own impatience, if truth be known. But, I'll try and stick with it and spend more time trying to learn the best places to implement the commands in some logical fashion.
Glad to hear it, skipper. Though don't forget to actually enjoy doing it once in a while. I love seeing it as nerdy detective work, considering myself a bit of a data sleuth who scrutinizes the trail of clues to eventually find the suspect. If you work at it as just that - a trail of evidence - then that can help with your analysis. Sometimes trails end abruptly and you'll want to start back at the crime scene and follow another trail, or you just need to reevaluate the whole thing. Sometimes even the most obvious potential suspect can end up being the most misleading.
It's a rather fascinating venture, and seeing the pieces fit together is always a satisfying experience. Once you learn to deep dive further into a crashdump to find more specific yet relevant details, it becomes more and more exciting - like learning to play an instrument and getting to the point where you can play your favorite songs.
Of course, like playing an instrument, this isn't something for everyone. If all this stuff does is end up making you tear your hair out and down Advil like it's M&Ms, then perhaps you're better off leaving this to someone else and focusing on other fields of study that cater more to your aptitude and interests. Nobody will hold it against you on that. We already know how difficult it is, so to see someone try to begrudgingly work through it with no satisfaction would be a living hell for them!