Win 7 64 RAM filling up. Memory leak outside windows?

[timeless]

since turning off superfetch (until l get more ram) my memory usage has gone way way down, especially after watching video files, however l need to watch more to get a definitive result.. will keep peeps posted (wish l didnt have to turn if off tho but its working at the moment so l shall keep it under observation to see how things go)

tho one thing l will say, when l woke up ny systems idle usage was around 900meg, and after messing around with firefox for a few hours and messing around with some japanese games lm using no more than 1.9gig of my 3gig of memory

 

[Dietch]

Kill AVG and switch to Microsoft Security Essentials.

This.
I switched from avira antivir to MSE last week and it fixed my leak.

 

[timeless]

as previously mentioned, the tweaks l made by disabling superfetch have worked out well, when idling and not doing anything l idle at around 900meg and l notice no slowdown, the only things taking up the most ram are Bitdefender, Vuze, and Firefox before l had less than 900meg left to spare while running after watching video files but now l have a spare 2gb to play with out of 3 (that is until l can afford to splash out on 8gb sticks)

 

[jemcgarvey]

Randomly happened again. Loading a 700mb .mkv file. The file played eventually, but not until somehow consuming all 4gb like before.

 

[baarod]

Randomly happened again. Loading a 700mb .mkv file. The file played eventually, but not until somehow consuming all 4gb like before.

Please repeat test case with http://www.sevenforums.com/performa...memory-leak-outside-windows-4.html#post677586

And consider http://www.sevenforums.com/performa...memory-leak-outside-windows-4.html#post687354

It's either an errant codec or zplayer. I strongly suggest using Divx if only for it's Windows 7 Media Foundation native Matroska splitter which can utilize the trouble-free h.264 codec that comes with Windows. How about dumping CoreAVC for the short term and retesting?

 

[jemcgarvey]

Well I'd really appreciate to hear if anyone has learned anything about this memory rubbish. I haven't posted in months, but mostly because I had no reason to expect any more success at fixing it as with all the other issues that still plague windows after decades of "upgrades".

Like I said before, it has nothing to do with specific codecs or media players. I've reinstalled (win7 64 pro) about 3-4 times since the summer, and never even installed core avc, or zoomplayer. I've been using the basic vlc which has been improved enough to handle 1080p perfectly now.

However, the RAM still is constantly at high usage levels, and now more than ever, everything from Notepad to Firefox suffers intermittent bouts of (not responding) if I dare to do so much as "click" and type more than three letters in a row. How dare I!

Notes:
Just ran sfc /scannow and got nothing. I'm running 4GB of DDR3, 32GB ssd with 2GB of pagefile, with 10GB of pagefile on a 2TB partition at the beginning of the drive. All drives are recent SATA or SATA2 models, all have been chkdsk'd and defrag/optimised within the last month, a quick bench shows all are running at avg to normal read/write speeds (have accidentally jumped into PIO mode before). Windows is up to date, motherboard has drivers from ASUS website and GPU the latest ATi driver.
Edit: Superfetch was still Automatic by default, though the install drive is an OCZ SSD. Rebooting with it off...

 

[cluberti]

You should consider using perfmon, or using Process Explorer and VMMap to figure out exactly what type of resource is being used. Worst case scenario, getting a complete memory dump during times of high memory pressure would be another way. That would be more painful, but would work.

 

[jemcgarvey]

I have been using perfmon, as per my screenshots.

As far as VMMap, I can see that every process (about 60-70) has a great deal more "committed" than "in use" or "working set", the total of these "committed" quantities could equate to what I see in the performance tab of taskman, but there's no single offender. Firefox gets up to 300MB or so with average use (200MB working), but everything else is below that level (even vlc playing 15mbps 1080p video) except for a game.

I did swap the memory out for a 4x2GB set, which has temporarily alleviated the "crunch" of running out of memory with a 3D game running, but still does nothing to explain why 7 likes to use 2000MB for system processes, keep 1800MB in standby, and then yell at me when I try to run something requiring 30-40% of system memory...

 

[cluberti]

It should be giving reserved memory back to you, but actually committing memory at that clip seems quite odd. It might be worth getting a complete memory dump of the system once it gets bogged down, honestly, so we can take the memory manager apart for you.

 

[jemcgarvey]

This is the first dump I've recorded. Unfortunately I couldn't get the memory to fill up (I am still running the 8GB set), but the system was "not responding" massively, so I forced a crash dump (ctrl+scroll lock).

http://rapidshare.com/files/436790028/MEMORY.zip

If there's a better host for distributing this size of file (100MB+) let me know please. Thank you!

 

[cluberti]

Preliminary update, but it looks like (so far) that the AVG filter driver has put a ton of disk IRPs into a completion pending state, meaning your disk queue is backing up (thus being a likely culprit for a hang scenario). I still need a bit of time to go through all of these stuck IRPs and make sure it's not just a victim of something else backing up the queue, but so far, it looks like AVG as the cause of your "massive not responding" . I'll post back with more in an hour or three - if you catch this before I update, it might be worth completely uninstalling AVG and retesting, as that is likely to restore some sanity to your system (this is one of the many reasons that most of us here do not recommend AVG antivirus software).

 

[cluberti]

So, ultimately, yes, it appears that your disk queue was growing. You have a few hundred threads on the system that look like this, waiting on NTFS flush routines to write data to the disk:

// This thread is doing non-cached I/O, meaning this *has* to be written to disk, not to the cache:
3: kd> !thread fffffa8006d27680 
THREAD fffffa8006d27680 Cid 0004.0024 Teb: 0000000000000000 Win32Thread: 0000000000000000 WAIT: (Executive) KernelMode Non-Alertable
fffff88003393258 NotificationEvent
IRP List:
fffffa800799a010: (0006,0550) Flags: 00060043 Mdl: fffffa800751a010
Not impersonating
DeviceMap fffff8a000008b30
Owning Process fffffa8006ca8b30 Image: System
Attached Process N/A Image: N/A
Wait Start TickCount 81474 Ticks: 4 (0:00:00:00.062)
Context Switch Count 7193 IdealProcessor: 0 
UserTime 00:00:00.000
KernelTime 00:00:00.280
Win32 Start Address nt!ExpWorkerThread (0xfffff80002cde850)
Stack Init fffff88003393db0 Current fffff88003392c80
Base fffff88003394000 Limit fffff8800338e000 Call 0
Priority 13 BasePriority 13 UnusualBoost 0 ForegroundBoost 0 IoPriority 2 PagePriority 5
Child-SP RetAddr : Args to Child : Call Site
fffff880`03392cc0 fffff800`02cd8992 : 00000000`0003aa00 fffffa80`06d27680 00000000`00000000 00000000`00000000 : nt!KiSwapContext+0x7a
fffff880`03392e00 fffff800`02cdacff : fffff880`033932f0 fffffa80`07d10950 00000000`00000000 fffff880`03393030 : nt!KiCommitThreadWait+0x1d2
fffff880`03392e90 fffff880`0123605f : fffffa80`07da9b00 00000000`00000000 fffff8a0`0bb2c100 fffffa80`07f5e100 : nt!KeWaitForSingleObject+0x19f
fffff880`03392f30 fffff880`0123ca12 : fffff880`033932f0 fffffa80`0799a010 fffffa80`07f5e180 fffff8a0`0bb2c140 : Ntfs!NtfsNonCachedIo+0x23f
fffff880`03393100 fffff880`01241413 : fffff880`033932f0 fffffa80`0799a010 fffff880`03393401 fffff880`03393401 : Ntfs!NtfsCommonWrite+0x872
fffff880`033932c0 fffff880`010e123f : fffffa80`0799a518 fffffa80`0799a010 fffffa80`079e7d30 00000000`00000000 : Ntfs!NtfsFsdWrite+0x1c3
fffff880`03393540 fffff880`010df6df : fffffa80`07d838c0 fffffa80`0b8a3dd0 fffffa80`07d83800 fffffa80`0799a010 : fltmgr!FltpLegacyProcessingAfterPreCallbacksCompleted+0x24f
fffff880`033935d0 fffff880`03bd2803 : fffffa80`0799a010 fffff800`02cbdac2 f8a00514`7fd00400 f8a00514`7fd80400 : fltmgr!FltpDispatch+0xcf
fffff880`03393630 fffff800`02cc02ff : fffffa80`0a3af060 fffffa80`0799a010 fffffa80`0751a010 fffff800`02e4be80 : AVGIDSFilter+0x5803
fffff880`033936d0 fffff800`02cbf987 : 00000000`00000f00 fffff880`03393c58 fffffa80`035310f0 00000000`00000000 : nt!IoSynchronousPageWrite+0x24f
fffff880`03393750 fffff800`02cbd804 : fffff8a0`0cfbd7a0 fffff8a0`0cfbd9d8 fffffa80`071d5f90 fffffa80`071d5f90 : nt!MiFlushSectionInternal+0xa58
fffff880`03393990 fffff800`02cbd259 : 00000000`00013e48 00000000`00000000 00000000`00047000 00000000`00000000 : nt!MmFlushSection+0xa4
fffff880`03393a50 fffff800`02cc494b : fffffa80`079420c8 fffff800`00000001 00000000`00000001 fffff800`00047000 : nt!CcFlushCache+0x5e9
fffff880`03393b50 fffff800`02cc5520 : 00000000`00000000 fffff880`03393c58 00000000`00000000 00000000`00000000 : nt!CcWriteBehind+0x1eb
fffff880`03393c00 fffff800`02cde961 : fffffa80`06d21ab0 fffff880`0122d270 fffff800`02ed8140 fffff800`00000000 : nt!CcWorkerThread+0x1c8
fffff880`03393cb0 fffff800`02f75c06 : 00000000`00000000 fffffa80`06d27680 00000000`00000080 fffffa80`06ca8b30 : nt!ExpWorkerThread+0x111
fffff880`03393d40 fffff800`02cafc26 : fffff880`03164180 fffffa80`06d27680 fffff880`0316efc0 00000000`00000000 : nt!PspSystemThreadStartup+0x5a
fffff880`03393d80 00000000`00000000 : fffff880`03394000 fffff880`0338e000 fffff880`033939f0 00000000`00000000 : nt!KxStartSystemThread+0x16
 
// The IRP associated with this thread, showing pending an NTFS completion routine:
3: kd> !irp fffffa800799a010
Irp is active with 16 stacks 15 is current (= 0xfffffa800799a4d0)
Mdl=fffffa800751a010: Irp count=00000002: Thread fffffa8006d27680: Irp stack trace. 
cmd flg cl Device File Completion-Context
[ 0, 0] 0 0 00000000 00000000 00000000-00000000 
Args: 00000000 00000000 00000000 00000000
[ 0, 0] 0 0 00000000 00000000 00000000-00000000 
Args: 00000000 00000000 00000000 00000000
[ 0, 0] 0 0 00000000 00000000 00000000-00000000 
Args: 00000000 00000000 00000000 00000000
[ 0, 0] 0 0 00000000 00000000 00000000-00000000 
Args: 00000000 00000000 00000000 00000000
[ 0, 0] 0 0 00000000 00000000 00000000-00000000 
Args: 00000000 00000000 00000000 00000000
[ 0, 0] 0 0 00000000 00000000 00000000-00000000 
Args: 00000000 00000000 00000000 00000000
[ 0, 0] 0 0 00000000 00000000 00000000-00000000 
Args: 00000000 00000000 00000000 00000000
[ 0, 0] 0 0 00000000 00000000 00000000-00000000 
Args: 00000000 00000000 00000000 00000000
[ 0, 0] 0 0 00000000 00000000 00000000-00000000 
Args: 00000000 00000000 00000000 00000000
[ 0, 0] 0 0 00000000 00000000 00000000-00000000 
Args: 00000000 00000000 00000000 00000000
[ 0, 0] 0 0 00000000 00000000 00000000-00000000 
Args: 00000000 00000000 00000000 00000000
[ 0, 0] 0 0 00000000 00000000 00000000-00000000 
Args: 00000000 00000000 00000000 00000000
[ 0, 0] 0 0 00000000 00000000 00000000-00000000 
Args: 00000000 00000000 00000000 00000000
[ 0, 0] 0 0 00000000 00000000 00000000-00000000 
Args: 00000000 00000000 00000000 00000000
>[ 4, 0] 0 e0 fffffa8007f5e030 fffffa800b8a3dd0 fffff880012384f4-fffff88003393250 Success Error Cancel 
\FileSystem\Ntfs Ntfs!NtfsMasterIrpSyncCompletionRoutine
Args: 00047000 00000000 00ff4000 00000000
[ 4, 0] 0 0 fffffa8007f5e030 fffffa800b8a3dd0 00000000-00000000 
\FileSystem\Ntfs
Args: 00047000 00000000 00ff4000 00000000
 
// Waiting on a write of a file stream to the volmgr driver:
3: kd> !fileobj fffffa800b8a3dd0
Device Object: 0xfffffa8007ce0cd0 \Driver\volmgr
Vpb: 0xfffffa8007cdf9a0
Access: Read Write SharedRead SharedWrite SharedDelete 
Flags: 0x40100
Stream File
Handle Created
FsContext: 0xfffff8a00bb2c140 FsContext2: 0x00000000
CurrentByteOffset: 0
Cache Data:
Section Object Pointers: fffffa80079420c8
Shared Cache Map: fffffa800798e910 File Offset: 0 in VACB number 0
Data at offset 0 not mapped

All of the above threads on the system in that state are waiting on this IRP in the AVG filter driver:

// For some reason, the AVG TDI (network) driver is pending, holding up AFD, holding up volmgr, holding up NTFS:
3: kd> !irp fffffa80`06d5d300
Irp is active with 1 stacks 1 is current (= 0xfffffa8006d5d3d0)
 No Mdl: No System Buffer: Thread 00000000:  Irp stack trace.  
     cmd  flg cl Device   File     Completion-Context
>[  f, 6]   0 e0 fffffa80080ce9f0 fffffa8007441f20 fffff88002f6c450-fffffa8006ec18b0 Success Error Cancel 
        \Driver\Avgtdia afd!AfdRestartAbort
   Args: 00000002 00000000 00000000 00000000
 
3: kd> !drvobj fffffa80`080d2060
Driver object (fffffa80080d2060) is for:
 \Driver\Avgtdia
Driver Extension List: (id , addr)
Device Object list:
fffffa80080f0040  fffffa80080f3b00  fffffa80080f3d10  fffffa80080ce9f0
fffffa80080cec00  fffffa80080d5040  fffffa80080f09f0  
 
3: kd> lmvm avgtdia
start             end                 module name
fffff880`02e62000 fffff880`02ec3000   avgtdia    (no symbols)           
    Loaded symbol image file: avgtdia.sys
    Image path: \SystemRoot\system32\DRIVERS\avgtdia.sys
    Image name: avgtdia.sys
    Timestamp:        Tue Nov 09 15:13:33 2010 (4CD9AB6D)
    CheckSum:         0006643C
    ImageSize:        00061000
    Translations:     0000.04b0 0000.04e4 0409.04b0 0409.04e4

At this point, I'd remove AVG and see what happens - I'm not sure I'd replace it with anything just yet (you're still testing), but long term (post-testing) using MSE instead would be a good choice.

 

[jemcgarvey]

I had switched to MSE before and was under the impression that it had caused a memory problem, but I could just be mistaken in my memory. I have been watching I/Os and noticed that AVG keeps adding processes that send incessant I/O requests. I'll scrub it off and see what happens.

 

[jemcgarvey]

Well thanks cluberti, it seems that removing AVG has lightened my load quite a bit, as far as hang ups and "not responding". It's only been a few days but it's only hanging when the network is bogged down with other things.

This doesn't really have to do with the memory leak, but thanks nonetheless. I haven't had the memory problem since the increase in ram, and though it doesn't exonerate the memory controller in the least (usage is still high), it lets me breathe.

Anyway, if it manages to constrict me again, I'll be right back!

Cheers.

 

[cluberti]

If you want to continue tshooting the memory usage, that can be done. Let us know when you decide to go further with it. It will take some tools and debugging over time, of course.

 

[jemcgarvey]

Well I'm a student, so I don't have time when I want it, but I can work through it at a "chess by mail" pace.

I mentioned I haven't had any problems, but I notice this:

As I'm running chkdsk on the second 2TB in a row (one at a time). I know, I'm running two defrags as well, but the ram usage is the question, and that's definitely almost 7GB in use by one chkdsk routine.

 

[cluberti]

Yeah, that's a ton of usage by the explorer process (and it is committed, not just reserved). It'd be interesting to see a trace of explorer.exe at the time it looks like that (dumps over about a 30 minute period), although using DebugDiag on Win7 x64 is still hit or miss. If you could download/install it, create a memory/leak rule for explorer.exe, and take full dumps of that over about 30 minutes, that would be the best way to start.

 

[logicearth]

...As I'm running chkdsk on the second 2TB in a row (one at a time). I know, I'm running two defrags as well, but the ram usage is the question, and that's definitely almost 7GB in use by one chkdsk routine.

That is by design. There was a big whole stink about it before Win7 was released. But chkdsk will and does take as much memory as it possibly can. It does this to repair a disk much faster. It only does this when you use the "/r" switch (Repair) otherwise it doesn't utilize memory like that.

Not a memory leak for chkdsk.

 

[jemcgarvey]

Yeah in this instance I didn't run into any instability, and taskmanager is honest about how much ram is going to explorer, but I was a little surprised to see it, as I've never even had more than 4GB before.

 

[chrysalis]

What are you all basing these "leaks" on? What memory metric provided by what authority? Just because "Free" memory in Task Manager goes down does NOT mean you have a leak.

Here is how I define a leak.

eg. my IE8 leaks.

If my ram usage is higher then expected and I feel like diagnosing I will shutdown apps one at a time and wait for the big jump downwards. Often IE8 is leaking badly.

eg. task manager will report iexplore.exe using say 250meg ram, I shut it down and suddenly I have freed up a gig of ram.

The high ram usage also could be cache, disabling superfetch may help on that.