6GB of Memory Used... No Idea Where it Went!

H2SO4 · 27 Oct 2009

brianng said:

Hi H2SO4,
I dont know if any memory optimizer in the market use a proper way to clean-up RAM.

But me and my team tried to implement the memory optimization in a good way.

Thank you for taking the time to respond. I'm quite interested in your team's design solutions to these vexing problems.

brianng said:

...The app scans for all scannable processes in the system and attempt to empty as many pages as possible from the working set.

So what effect does reducing the working set of a process have on the performance of that process? Is there likely to be any downside?

brianng said:

WIndows manages the memory itself by default but it doesnt automatically change the min and max working set which is located by applications.

So your application steps in and sets a minimum and maximum working set size for each process? What mechanism does it use to try to reduce the working set of a process once it gets close to its set maximum?

brianng said:

And this can lead to memory leak in the system.

I'm not sure I follow - does the lack of control by the OS (as you described it) lead to a memory leak? If not, please help me to understand.

brianng said:

However Windows has a reason to do so, you can read more about this on MSDN.

A reason to do what exactly? Thank you for your kind explanation!

Teerex · 27 Oct 2009

Well.. You did a good job of packing all those mostly unnecessary free tweaks / tutorials floating around, into one well designed shareware app.

Your pricing is not to steep.. Well, I wish you all the luck in the world. Someone might think it worth 10/15 bucks, and that'll be good 4 u. I don't, but that's because I, like, know stuff.

H2SO4 · 27 Oct 2009

brianng said:

...It scans, find freeable working set and tell the memory management to free the memory.

How does your app "tell the memory management to free the memory"? Can you elaborate on that mechanism?

H2SO4 · 27 Oct 2009

brianng said:

H2SO4, I cant tell you that :) it is a complex story and mechanism.

Awww :)

Being thus forced to rely on my own inadequate understanding if I was going to learn more about this "memory optimisation" business, I took the liberty of downloading the trial version of Win7Zilla and having a little look.

After kicking off a Perfmon log, I attached a lil old utility called a "debugger" (I read about it in MSDN) to the Win7zilla process. On a hunch, I set a breakpoint on kernel32!VirtualAlloc before invoking Win7zilla's "extreme optimisation" routine:

Code:

Breakpoint 0 hit
KERNEL32!VirtualAlloc:
76fa1553 8bff            mov     edi,edi
0:005:x86> kv
*** ERROR: Module load completed but symbols could not be loaded for C:\Win7Zilla\install\bin\Win7Zilla.exe
ChildEBP RetAddr  Args to Child              
WARNING: Stack unwind information not available. Following frames may be wrong.
0344febc 00503401 00000000 00400000 00003000 KERNEL32!VirtualAlloc
0344ff60 005082e5 00000032 4ab64696 00000000 Win7Zilla+0x103401
0344ff88 76fa3f39 02cc2158 0344ffd4 77920409 Win7Zilla+0x1082e5
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for ntdll.dll - 
0344ff94 77920409 02cc2158 3770f085 00000000 KERNEL32!BaseThreadInitThunk+0x12
0344ffd4 779203dc 00508250 02cc2158 00000000 ntdll_778d0000!RtlInitializeExceptionChain+0x63
0344ffec 00000000 00508250 02cc2158 00000000 ntdll_778d0000!RtlInitializeExceptionChain+0x36
0:005:x86> ? 400000
Evaluate expression: 4194304 = 00400000
0:005:x86> ? 3000
Evaluate expression: 12288 = 00003000
0:005:x86> * first arg = 0 = don't care where in memory 
0:005:x86> * second arg = 4MB chunk = requested alloc size
0:005:x86> * third arg = 0x3000 = MEM_RESERVE | MEM_COMMIT

So Win7zilla's "extreme optimiser" had just requested 4MB of MEM_RESERVE|MEM_COMMIT memory anywhere inside it's own VM space. Fascinating :)

By now, I was suspecting it may do that a few more times, so I set up a pseudo-register to keep track of the Win7zilla VirtualAlloc running total:

Code:

breakpoint 0 redefined
0:005:x86> bu KERNEL32!VirtualAlloc ".echo ====== Another VirtualAlloc() call =========; kb5; r$t1 = $t1 + (esp+8); g"
breakpoint 0 redefined

Amazingly, Win7zilla then called VirtualAlloc >120 times more! The total amount requested by the time the "optimisation" fest was over topped 500MB.

The attached perfmon log clearly shows the Win7zilla process swelling up in memory, only to immediately release it all again. The only effect is to make the OS trim the working sets of all other (real) processes to make room for the Win7zilla "puffer fish" act. Of course, that just slows down the real apps because their memory has been pushed out to disk to make room for Win7zilla. It's the garden variety cheap 'n' nasty "memory optimiser" reporting trick. It looks like there's more memory available when the "optimiser" is finished, but that's because it has forced out the real apps to make an empty hole.

Over time, the OS will read back into RAM the real app stuff you caused to be paged out to disk, but until then the hapless user of your "optimiser" will suffer somewhat degraded performance.

Do you have anything to say to put a positive spin on all this?

torrentg · 27 Oct 2009

H2SO4 said:

On a hunch, I set a breakpoint on kernel32!VirtualAlloc before invoking Win7zilla's "extreme optimisation" routine:

That's some hunch just having read about it on MSDN and all lol. Can you please give me your best guess at tonight's lottery drawing numbers?

Teerex · 27 Oct 2009

Geeze, I thought it was only about unloading dlls, but it's waaaaaaaaaay cooler. Thanx a bunch for reveealing da cool, H2SO4.

Teerex · 27 Oct 2009

brianng said:

Our Quick Optimization method DID NOT force Windows for memory allocation!

It asked for it, nicely?

Sparatan117 · 27 Oct 2009

avery said:

So, I have the strangest problem I've ever run into. When downloading certain types of files that are of a certain size (I've found .ISOs bigger than 4GB to be the problem), my computer completely locks up. It runs out of memory. Task manager shows 99% memory used, but when I look at processes, and add up all the memory being used, it's barely a GB of mem. This doesn't happen with big movies, only ISOs. This happens no matter what browser I use to download, and even happens when I do a basic WGET in cygwin.

Any suggestions/ideas? Any way to see what's sappin' my memory?

I didn't see anyone actually answer your question So i'm just making sure you did.

If not what program are you using for the iso's

solar · 27 Oct 2009

thx for valuable input H2S04, I appreciate the look into such things and very valuable. thanks for making me aware to think about such claims/programs (I wasn't interested in itin this case as I tend to rather trust tested programs I notice a certain 'community' I will trust in is using it a lot or giving it the green flag. Simply because of things like you investigated. cheers again mate. :)

solar

H2SO4 · 27 Oct 2009

brianng said:

H2SO4 you are really a strong acid. You are right at some points. Can you debug Quick Optimize? I appreciate your comment and time.

Sincerely, I'm not sure whether I understand that statement. Either:

a) You already knew perfectly well that Win7zilla's "extreme optimisation" is a con which would only slow things down (and not a particularly original con either) and now you're bluffing and pretending that "quick optimize" is somehow substantially better in the hope that it'll be sufficiently complex to prevent me from revealing what it does quite so easily.

OR

b) You're a Win7zilla employee, perhaps not one of the developers, and the info I posted about the extremely negative operation of the "extreme optimise" button came as a genuine and unwelcome surprise to you.

If it's (a), no thanks. Without false modesty, I'd be able to discover how any aspect of your app functions under the hood, given time. Since one of the "memory optimisation" modes has been proven to be a con, I don't see what purpose it would serve for me to spend time analysing the mechanism(s) of the other mode.

On the other hand, if the reason for your current request is closer to (b), then I'll do it, but not now - real life intervenes, plus I cringe at some of the stuff I did in that log last night while I was dog tired.

As Teerex said, Win7zilla is a decent-looking utility for controlling registry values which are documented and exposed anyway, and you've got a perfect right to try to earn money by selling "tweaking" software. Where it gets reprehensible is when your non-free product becomes a confidence trick. I resent that type of "tax on the gullible", and I especially resent it when offered up on my fav forum as part of a supposed solution to the OP's problem.

And now, I hope the OP comes back to continue troubleshooting :)