Solved Automatically generate crash dump after AppCrash

seismo · Apr 27, 2010

Hello,
I'm on this problem since a few days, searched google for hours, and also used your forum-search, but apparently there is no solution for my problem.

First of all, my PC runs Windows 7 Enterprise x64.
So, I have a custom application which generates a StackOverflowException.
It's configured, that the application starts completely, waits a minute and then generates the crash.

My Problem is, that there is no Crash Dump generated in C:/Windows/PCHEALTH/ERRORREP/QSIGNOFF (I found this path in my recherches, iirc is that the place where i can find the dumps).

Maybe someone of you can give me the info, how i am able to automatically generate such dumps instantly after the app crashes.

sorry for my not-so-good english,

seismo

EDIT:

I just found an article Windows and .NET Programming : Collecting crash dumps which should solve my problem, but in fact it does absolutely nothing...

this description fits exactly my problem, but doesn't solve it:

What to do when the crashing process will be started in the future

An application crash is not always as easily reproducible as the ones from the CrashMe application. The special case when you know that a certain application will crash but don’t know under which circumstances can be handled in at least two ways:
a) Configuring the Windows Error Reporting to collect a customized dump when an application crashes. Here is a sample how to configure CDB to get dumps when any application crashes in Windows:
Set the following registry values of the key: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AeDebug
Debugger = "C:\Debuggers\cdb.exe" -p %ld -e %ld -g -c "~*k;.dump /mfh /u C:\Dumps\crash.dmp;.kill;qd"
Auto = “1”
Notes:
-“C:\Debuggers\cdb.exe” is the path to the CDB debugger, which might vary depending on the installation path of Debugging Tools for Windows.
-%ld will be provided by Windows and represents the PID of the process where the 2nd chance exception occurs.
- After the crash is collected, the crashing process is killed (this is specified by passing a custom command to CDB using the –c switch).

b) Monitoring for a certain process to start and to crash:

adplus.exe -crash -pmn notepad -o C:\Dumps

The parameter passed to adplus in –pmn is the name of the process to monitor and the parameter after –o is the directory where the DMP files will be collected.

I don't want to use adplus, because of performance reasons, so i just want to have a dump created whenever a program crashes.

EDIT2:

there is a tool called "User mode process dumper 8.1" (http://www.microsoft.com/downloads/...41-6A87-40C8-BF69-28AC08570B7E&displaylang=en) which is able to create dumps if a special process throws an exception or crashes. Unfortunately it cannot be installed under windows 7. I tested it under XP and it worked fine, so maybe someone can help here...

seismo · Apr 28, 2010

push (new information added yesterday)

jcgriff2 · Apr 28, 2010

Hi -

Items that come to mind. . .

1. Have you closely followed the crashing app and know for a fact that it crashes as T+60? ...and that it writes a dump file? Possibly the crash is being intercepted by a 3rd party dbghelp.dll..? For example, if the crashing app is running under Firefox, FFox's dbghelp.dll may grab the crash from Windows 7 and produce its own dump file.

- use SysInternals Process Explorer and watch real-time (like TaskMgr ++)
- use SysInternals Process Monitor - use filter to watch the app itself and the crash reporting. ProcMon will show you all HDD activity.

They must be executed at elevated admin level - RIGHT-click, "Run as Administrator"

See SysInternals Suite at Microsoft TechNet for additional information on these stand-alone executables.

Check for information on the crashing app - 2, 3, 4 -

2. Check Problem Reports -
START | type view all | "View all problem reports" | 2x-click on line item for additional crash information

3. Check Event Viewer
START | eventvwr.msc | Custom Views| Administrative Events

4. Check Reliability Monitor
START | perfmon /rel

5. Are you running the app in question at an elevated administrative level? RIGHT-click, "Run as Administrator"

6. Did you check the Virtual Store for the dump?

Code:

[FONT=lucida console]c:\users\<username>\appdata\local\VirtualStore [/FONT]

--> Common file and registry virtualization issues in Windows Vista or in Windows 7

7. Use WHERE command to search for dump file -
START | type cmd.exe | RIGHT-click on cmd.exe | "Run as Administrator" | type/ paste the following -

Code:

[FONT=lucida console]where /r c:\ /f /t *.*dmp > 0 & start notepad 0[/FONT]

8. Use the WHERE command - search for CAB files. The dump(s) and corresponding TXT &.or XML files may be CAB'd and in the reporting/ transmission que somehwere

9. Check Windows Error Reporting (WER) directories - your dumps & info may be there

- My observations of a typical WER layout -

Code:

[FONT=lucida console]─────WER[/FONT]
[FONT=lucida console]├───ReportArchive[/FONT]
[FONT=lucida console]│   │   store.lock[/FONT]
[FONT=lucida console]│   │[/FONT]
[FONT=lucida console]│   ├───Report017f99a1[/FONT]
[FONT=lucida console]│   │       Report.wer[/FONT]
[FONT=lucida console]│   │       Report.cab[/FONT]
[FONT=lucida console]│   │ ├───Report.wer[/FONT]
[FONT=lucida console]│   │ ├───AppCompat.txt               [/FONT]
[FONT=lucida console]│   | ├───memory.hdmp     [/FONT]
[FONT=lucida console]│   │     [/FONT]
[FONT=lucida console]└───ReportQueue[/FONT]
[FONT=lucida console]        store.lock[/FONT]

- WER locations -

Code:

[FONT=lucida console]C:\Users\All Users\Microsoft\Windows\WER (1)[/FONT]
 
[FONT=lucida console]C:\Users\Administrator\AppData\Local\Microsoft\Windows\WER (2)[/FONT]
 
[FONT=lucida console]C:\Users\<username>\AppData\Local\Microsoft\Windows\WER[/FONT]
 
 
[FONT=lucida console]─────────────────────────────────────────────[/FONT]
[FONT=lucida console](1) = same folder as C:\ProgramData\Microsoft\Windows\WER[/FONT]
[FONT=lucida console](2) = "Administrator" user name only present if the Hidden Admin account active and used[/FONT]

Inside the WER directory, you can expect to find these directories (ERC in \users only) -

Code:

[FONT=lucida console]04/28/2010  04:43    <DIR>          ERC[/FONT]
[FONT=lucida console]04/27/2010  04:27    <DIR>          ReportArchive[/FONT]
[FONT=lucida console]04/26/2010  08:10    <DIR>          ReportQueue[/FONT]

Thye will be in blue as file attributes are "c" = compressed

Inside "ReportArchive" -

Code:

[FONT=lucida console]\---ReportArchive[/FONT]
[FONT=lucida console]|   |       [/FONT]
[FONT=lucida console]|   +---AppCrash_DllHost.exe_8879601beeeb3cebfbf1ef8579c35a2ed9f4a3d_02e2a710[/FONT]
[FONT=lucida console]|   |       Report.wer[/FONT]
[FONT=lucida console]|   |       [/FONT]
[FONT=lucida console]|   +---AppCrash_iexplore.exe_77a6d928d6d799962d8f617f7185ddea51912e_02030734[/FONT]
[FONT=lucida console]|   |       Report.wer[/FONT]
[FONT=lucida console]|   |       [/FONT]
[FONT=lucida console]|   +---AppCrash_windbg.exe_917d996ec02c36be0a6f6582e1f8c3c349e0_06645ab2[/FONT]
[FONT=lucida console]|   |       Report.wer[/FONT]
[FONT=lucida console]|   |       [/FONT]
[FONT=lucida console]|   +---Critical_6.1.7600.16385_be86ea8c9ed8f2759357331b1943c3f4eeaee7_01289d5a[/FONT]
[FONT=lucida console]|   |       Report.wer[/FONT]
[FONT=lucida console]|   |       [/FONT]
[FONT=lucida console]|   +---Critical_{11111111-1111-1_81b558e7b458c1c57d5a6e4733dbfdcbc1ce1c1_13e80425[/FONT]
[FONT=lucida console]|   |       Report.wer[/FONT]
[FONT=lucida console]|   |       [/FONT]
[FONT=lucida console]|   +---NonCritical_6.1.7600_56e1975b826845421f2ab6e4a688ec57ef8df_026041e3[/FONT]
[FONT=lucida console]|   |       Report.wer[/FONT]
[FONT=lucida console]|   |       [/FONT]
[FONT=lucida console]|   +---NonCritical_6.1.7600_56e1975b826845421f2ab6e4a688ec57ef8df_026eb52f[/FONT]
[FONT=lucida console]|   |       Report.wer[/FONT]
[FONT=lucida console]|   |       [/FONT]
[FONT=lucida console]|   \---NonCritical_x64_f7e69524d9cb28c827a39669d52f42e87cd2743_02e4b256[/FONT]
[FONT=lucida console]|           Report.wer[/FONT]

Inside "ReportQueue" - The WD- dumps in RED = "Watch Dog" dumps generated by 0x117 (TDR) Live Kernel Events. The dumps you are looking for could be named anything, but should have file ext = .dmp, .hdump or .mdump -

Code:

[FONT=lucida console]\---ReportQueue[/FONT]
[FONT=lucida console]  +---Kernel_0_0_cab_04d92d4c[/FONT]
[FONT=lucida console]  |       Report.wer[/FONT]
[FONT=lucida console]  |       [COLOR=red]WD-20100325-0923.dmp[/COLOR][/FONT]
[FONT=lucida console]  |       WER-17898135-0.sysdata.xml[/FONT]
[FONT=lucida console]  |       WER2D4C.tmp.WERInternalMetadata.xml[/FONT]
[FONT=lucida console]  |       [/FONT]
[FONT=lucida console]  +---Kernel_0_0_cab_07fd327a[/FONT]
[FONT=lucida console]  |       Report.wer[/FONT]
[FONT=lucida console]  |       [COLOR=red]WD-20100325-0923-01.dmp[/COLOR][/FONT]
[FONT=lucida console]  |       WER-17903579-0.sysdata.xml[/FONT]
[FONT=lucida console]  |       WER326A.tmp.WERInternalMetadata.xml[/FONT]
[FONT=lucida console]  |       [/FONT]
[FONT=lucida console]  +---NonCritical_8024402c_a728027ac40dacf25be0b7dcff722b28db819_0e2063e5[/FONT]
[FONT=lucida console]  |       Report.wer[/FONT]
[FONT=lucida console]  |       [/FONT]
[FONT=lucida console]  +---NonCritical_8024402c_a728027ac40dacf25be0b7dcff722b28db819_cab_06c08ead[/FONT]
[FONT=lucida console]  |       client_manifest.txt[/FONT]
[FONT=lucida console]  |       Report.wer[/FONT]
[FONT=lucida console]  |       [/FONT]
[FONT=lucida console]  +---NonCritical_Microsoft_f8c48451f768f83c338e744dd14368140cecf28_cab_008240ed[/FONT]
[FONT=lucida console]  |       Report.wer[/FONT]
[FONT=lucida console]  |       Report.wer.tmp[/FONT]
[FONT=lucida console]  |       {2930EFF1-D17C-4788-9EB5-3E7A0705C54C}-WER-04152010-1841.etl[/FONT]
[FONT=lucida console]  |       [/FONT]
[FONT=lucida console]  +---NonCritical_speedfan.exe_6221b158394fd03e5a99fe5eccb9f34b6e94e6fd_cab_0ed0d406[/FONT]
[FONT=lucida console]  |       appcompat.txt[/FONT]
[FONT=lucida console]  |       Report.wer[/FONT]
[FONT=lucida console]  |       TabD37A.tmp[/FONT]
[FONT=lucida console]  |       [/FONT]
[FONT=lucida console]  \---NonCritical_Synaptics_v14_0__35ba694dfbec3292e3cc2aebe37fc2766385d_cab_08c6f2f5[/FONT]
[FONT=lucida console]          appcompat.txt[/FONT]
[FONT=lucida console]          Report.wer[/FONT]

10. I found a Microsoft kb on this topic, but please note it does not mention Windows 7 -
How to use the Debug Diagnostics tool to troubleshoot an IIS process that stops unexpectedly

11. You are Windows 7 x64, correct? Are you by any chance executing the crashing app from within an x86 app? If so, the crashing app would be utilizing whatever it needs from \windows\syswow64, not \windows\system32, due to x86 re-direction away from the x64 \system32. I don't know if this scenario comes into play here or not.

That is all I can think of for now.

Regards. . .

jcgriff2

.

seismo · Apr 28, 2010

Hi, first of all thank you for that detailed answer.

to your items:

1. sry i don't exactly understand what you mean with "crash as t+60". so i assume you mean that the application crashes in the future. The answer to that is yes. The application i want to monitor is a product of my company, and sometimes it creates a StackOverflowException. Now we want to debug the dump to find out the reason for that. It's programmed under .NET C#, and the server-part of it is installed as service. To test this case, i configured it, to create an StackOverflowException a few minutes after it started. So the program does not run under Firefox, and there is no custom dbghelp.dll used from our application.

2. Thank you for that, i didn't know this tool yet, there i can see that my application had an APPCRASH, and if i double-click this line, i have the option to "View temporary copies of this files" (i have a german version of windows7) where i suddenly found the dumps. these files didn't exist till i clicked there (files were extracted)

3. Checking Event Viewer was one of my first steps when i started trying to find out why no dumps are created, but for me as an amateur in this topic (and the fact that i'm new at windows 7) i found too few information.

4. I didn't know that tool yet, but i see it shows exactly those crashes i produced with my application.

6. Virtual Store is empty

7. Because i did what i described in 2., i'm able to see the dump-file-paths (which didn't exist before, i always looked in this directory for dumps)

9. I crashed the application a few times, so i have more AppCrash*-folders in my ReportQueue, but there are just Report.wer files in those different folders with blue fontcolor, except the one, i clicked when i did 2., there are all files in it, also the dump files.

10. thx for that lecture, i just bookmarked it for future research.

11. yes it's a Windows 7 x64, but all the things you described work very well...

In the end, i came to the conclusion, that i have to extract the dump files manually over the "View all problem reports" tool, and i get all i want.
Didn't think that this is so easy, when i think of my tries the last days:

- changing registry keys (and changing them back after figuring out that my changes didn't work)
- trying different tools, which all do the same: attach as a debugger, which reduces the performance of the application
- research research research

thank you very much for your help,
seismo

jcgriff2 · Apr 28, 2010

Hi -

You are very welcome. I am glad that my early-morning thoughts produced positive results for you.

For future reference, if you are going to "regedit" and know the likelihood exists for reversal of registry changes, create a system restore point first. Doing so will allow you to then run system restore, which will restore the registry (as well as most of Windows folder... ) to your starting point.

START | type create | "Create a Restore Point"

To restore the system using the restore point -

START | rstrui

There may be an easier way to get all of your dump files as well. It appears to me that since they are only available in WERCON (Problem Reports), they are stored in CAB files. Windows Explorer will never allow you to touch them. But 3rd party product I use daily can get right to the compressed CAB files and un-CAB them.

Altap Salamander v2.53 beta 1 running at elevated administrative level can do it all.

Download and install this --> http://www.altap.cz/ftp/salamand/as253b1.exe

It is from this site --> ALTAP - Download

Post-install, RIGHT-click on Altap icon, select "Run as Administrator". Go to the WER directories I listed in my last post, click on the "date" header to sort, 2x-click to open the compressed directory, find the CAB file and 2x-click on it. The CAB file will open right up. In the 2nd window, have a directory ready so you can copy out of the opened CAB file those files you want to further review/ dump to debug. Left-click 1x on one of the files in the opened CAB, press alt-a (or press spacebar) and the will turn RED. Then press F5 and the CAB'd files will be copied to the other directory.

One thing to keep in mind with Altap - the 2.53b1 version is x86, so if you venture into \windows\system32, you will actually be viewing the contents of \windows\syswow64, even though the header will say \system32. Notice a "directory" called \sysnative -- that is the x64 \system32 under an x86 app. Altap is working on an x64 version right now.

Finally - re: "T-60" -- I read and took the following literally . . .

It's configured, that the application starts completely, waits a minute and then generates the crash.

T+60 = 60 seconds after the app started running.

Regards. . .

jcgriff2

.

jcgriff2 · Apr 28, 2010

Apologies for double post, but I am curious about the Stack Overflow Exceptions you are encountering.

I see stacks in kernel memory dumps, but do know that a Stack Overflow occurs in user mode and would therefore produce a user-mode dump, which differs from the post-mortem kernel dumps I usually work with.

I know of 3 reasons for Stack Overflows in the limited number of user dumps I have looked at -
1. thread has used up the stack reserved for it
2. page file is maxed-out, so no additional pages can be committed to extend the stack
3. system resource allocation issue - the page file is being extended at the time the thread needs additional pages (system is "busy")

Are there others?

What is the exception code - or is that what you are looking for?

Thanks. . .

jcgriff2

.

seismo · May 11, 2010

sorry that i didn't answer for such a long time, i wasn't able to check here.

thank you for your advices, so i was able to create crash dumps.
you asked for the exception code, so i have to say, that I don't really know which Exception Code it is,
because it was just a task for me, given from my coworkers, to find out how to generate crash dumps.
but with your help i was able to fulfill that

regards
seismo

jcgriff2 · May 14, 2010

That's fine.

I am glad the dump producing issue has been solved for you.

Thank you for posting back.

Have a good weekend.

jcgriff2

.

Solved Automatically generate crash dump after AppCrash

seismo

New member

My Computer

At a glance

seismo

New member

My Computer

At a glance

jcgriff2

Account closed

My Computer

At a glance

seismo

New member

My Computer

At a glance

jcgriff2

Account closed

My Computer

At a glance

jcgriff2

Account closed

My Computer

At a glance

seismo

New member

My Computer

At a glance

jcgriff2

Account closed

My Computer

At a glance

Similar threads

Solved Automatically generate crash dump after AppCrash

New member

My Computer My Computer

At a glance

New member

My Computer My Computer

At a glance

Account closed

My Computer My Computer

At a glance

New member

My Computer My Computer

At a glance

Account closed

My Computer My Computer

At a glance

Account closed

My Computer My Computer

At a glance

New member

My Computer My Computer

At a glance

Account closed

My Computer My Computer

At a glance

Similar threads

My Computer

My Computer

My Computer

My Computer

My Computer

My Computer

My Computer

My Computer