Too much memory on standby.....

Page 4 of 6 FirstFirst ... 23456 LastLast

  1. Posts : 26
    Windows 7 Home Premium 64bit
       #31

    logicearth said:
    Right now unless we know exactly what you are doing, it is impossible to help you.
    Please re-read my posts. I do not know how to be more clear.

    Running a slide show application (.EXE) crashes after about 15 minutes when there is ZERO Free memory.

    Running the same slide show application, runs to completion, multiple time, when there is a "lot" of Free memory available.
      My Computer


  2. Posts : 5,642
    Windows 10 Pro (x64)
       #32

    Then it is the application that is broken. Not Windows or its Standby Memory. The application must be doing something that is causing it to crash. Since I don't know want application it is, no idea what logs it makes.

    And it certainly is not because Windows is not releasing Standby memory quick enough. That is not an issue, Windows doesn't release Standby it just over writes it. Clearly something is going wrong with your application.
      My Computer


  3. Posts : 2,528
    Windows 10 Pro x64
       #33

    Ultimately, logicearth is correct - standby memory is effectively free memory if there's demand, as the contents of a "standby" page are pages that are not in a process' working set (actively touched memory), and that do not need a backing store to be removed from the working set either (memory that can go directly to the standby list can be temporary data, binaries from disk, freed memory, etc.). When a process makes an alloc request (not just a reservation), the memory manager initially looks to the zero list to fulfill the request (pages that have been freed and then completely zeroed out). If that list is empty, the memory manager then looks to the free list - if that list is empty, then standby pages are invalidated and used as free pages.

    Standby pages are pages that have been allocated by a process (or multiple processes), but they did not contain "private" data before the memory manager deemed them to be no longer in use and yanked them from a process' working set. Standby pages are pages that do not need to be backed before they're cleared from RAM, because they're temporary, freed, or part of the "shared" portion of the working set, and as such do not need to be saved (they're either still in memory as part of the shared portion of another process' working set, or the memory is indeed no longer in use by any processes).

    If the page was deemed able to be removed from the working set of a process, but was previously listed as part of the private portion of the working set (private working set memory is non-shared, as in it's data contents are unique to the process that used it), it's contents could be marked as not in use and pulled from a working set - but that page requires a backing store before being cleared. These pages are on the "modified" page list - they're still available to be used by other processes when the system is under memory pressure, but the contents of the pages would have to be put into a backing store (the paging file), then the page can be moved to the standby list where it could then be used as previously described (the modified page writer in the kernel is responsible for backing modified pages and then putting them on the standby list).

    So, if you have a large standby list, but are having issues with performance or applications misbehaving, then it's not likely the OS that's at fault. We'd need to know more about *how* the process was requesting memory, and what types of requests for memory it was making (is it making a request for a reservation? Is it actually making an alloc/malloc call to acquire RAM? Is it actually trying to write data to those pages?). Something's not right here, and given that the only way an application can actually talk to physical RAM is to use AWE (32bit only) or use a request to lock pages via a driver (which will fail if there's not enough memory for the request, btw), we'd need to know more to understand why the application in question has issues when Windows starts getting low on free or zero pages. However, again, since by default a process has *zero* understanding of Physical RAM in a box, and can only work with virtual address space (VA), then we'd need to know more. However, if you have adequate standby memory, with little to no "free" or "zero" memory, then it's either a filter driver (not likely, but possible) that's preventing the memory manager from being able to access memory, or the application is trying to directly access RAM versus going through the regular memory manager APIs. In either case, the memory manager *should* be clearing the standby list if free/zero are empty. If it's not, then either there's something installed on the system that's preventing it, or the app itself is trying to do memory management (and failing).

    Since we don't know why, we can only state that we don't know, but this is how it *should* work. If it isn't working that way, then we need to know more about how the app (or apps, in this case) are designed to request and allocate memory.
      My Computer


  4. Posts : 26
    Windows 7 Home Premium 64bit
       #34

    cluberti said:
    ... When a process makes an alloc request (not just a reservation), the memory manager initially looks to the zero list to fulfill the request (pages that have been freed and then completely zeroed out). If that list is empty, the memory manager then looks to the free list - if that list is empty, then standby pages are invalidated and used as free pages.

    Standby pages are pages that have been allocated by a process (or multiple processes), but they did not contain "private" data before the memory manager deemed them to be no longer in use and yanked them from a process' working set.
    Well stated and I concur !

    cluberti said:
    So, if you have a large standby list, but are having issues with performance or applications misbehaving, then it's not likely the OS that's at fault. We'd need to know more about *how* the process was requesting memory, and what types of requests for memory it was making ...
    .
    .
    .
    Since we don't know why, we can only state that we don't know, but this is how it *should* work. If it isn't working that way, then we need to know more about how the app (or apps, in this case) are designed to request and allocate memory.
    We will never know just exactly what the application is doing !

    I can only report what I have observed. Large Free list application runs to completion. Large Standby list no Free list, application crashes, but not immediately. I observed no changes in the size of the Standby or Free list while the application was running.

    Even odder (from my perspective) is that well after the application crashed, and other tasks completed (that caused the Standby list to grow), there was still a large Standby list and ZERO Free list.


    IMHO, an application programmer should not have to worry about how an OS manages memory. They should be confident that it will always "do the right thing" and provide them the virtual memory space needed to run. Or stated another way, a poor algorithm should cause an application to run poorly, not crash.
      My Computer


  5. Posts : 2,528
    Windows 10 Pro x64
       #35

    The problem begins if the application developer starts to stray from the beaten path. If this was a poor algorithm, everyone in the world would be complaining of it. The fact that it happens so infrequently (unfortunately for you it's happening to you) would indicate a problem with the app code, and not the OS. It's *possible* that you're hitting a bug, but it's not very probable. The OS isn't going to magically move pages from standby to the free list unless the zero page list is also empty. Standby pages could conceivably still be requested by the process that initially requested them, so standby pages will NOT go from standby to ANY list unless there are applications making alloc requests for memory (again, not just reservations, but allocs and then make an attempt to write to that requested page). In your case, the fact that you can start applications that should very obviously be requesting memory (the simple act of starting a process would do this, for things like default heap, loading included modules, etc) but seemingly are not would tend to indicate an app programming issue, and not an OS issue. Again, there *are* ways to do your own memory management without using the inbuilt APIs to request memory (especially if the application is written in C or ASM). I'm not saying that's what's happening, but.... that's what appears to be happening.
      My Computer


  6. Posts : 26
    Windows 7 Home Premium 64bit
       #36

    35+ years administering medium and large VM systems. I stand by my position.

    theoldwizard1 said:
    An application programmer should not have to worry about how an OS manages memory. They should be confident that it will always "do the right thing" and provide them the virtual memory space needed to run. Or stated another way, a poor algorithm should cause an application to run poorly, not crash.
      My Computer


  7. Posts : 2,497
    Windows 7 Pro 64 bit
       #37

    theoldwizard1 said:
    IMHO, an application programmer should not have to worry about how an OS manages memory. They should be confident that it will always "do the right thing" and provide them the virtual memory space needed to run. Or stated another way, a poor algorithm should cause an application to run poorly, not crash.
    Very true, provided some conditions are met.

    1. The application must follow the rules as established by the OS. When an application breaks the rules it must suffer the consequences, including crashing.

    2. The application must not ask for resources the system cannot provide. As an example, when allocating memory (virtual address space) it must not exceed the virtual address space or the commit limit.

    I am not including RAM as a resource. Unless the application is using something like AWE it need have no knowledge of how much RAM is in the system, how much is available, etc. This is managed by the system without application involvement.

    Unfortunately, sometimes application do violate riles #1 and #2 and the OS cannot always protect them from the consequences. The OS does have responsibility to see that application mistakes do not result in a system crash.
      My Computer


  8. Posts : 25,847
    Windows 10 Pro. 64/ version 1709 Windows 7 Pro/64
       #38

    I don't create programs or operating systems but to my understanding it is the program creators job to make their program work with the operating system they intend to use it with.
    Microsoft is not going to change Windows 7 to suite a particular program nor should they.

    Thousands if not millions of programs are made to work with Windows 7 so as just being a dummy like me I have to believe it can be done if the rules for making a program to be used on Windows 7 are followed.
      My Computer


  9. Posts : 26
    Windows 7 Home Premium 64bit
       #39

    LMiller7 said:
    Very true, provided some conditions are met.
    I disagree with both

    LMiller7 said:
    1. The application must follow the rules as established by the OS. When an application breaks the rules it must suffer the consequences, including crashing.
    The application must follow the rules of THE COMPUTER LANGUAGE IT IS WRITTEN IN, which provides the run-time environment! The OS should have very little, if any affect on the application, unless the application is using OS-specific procedures.

    If your statement was true, then what good would would application written in Java, Ruby or Python be ?

    LMiller7 said:
    2. The application must not ask for resources the system cannot provide. As an example, when allocating memory (virtual address space) it must not exceed the virtual address space or the commit limit.
    Only partially true. The application can ASK for any resource it wants. It is up to the OS to decide if the request and be fulfilled and in NOT somehow tell the application, "Hey, I can't do that !"



    Sadly most programmers out there to day have grown up in either the Windows or Un*x environment typically using the C programming language. IMHO, these do NOT lend themselves to educating the programmer to check the status of errors. Even if they do the error code usually lack the "robustness" to allow the application to handle the error in an "intelligent" manner.

    Sadly, many concept well defined and taught by now "extinct" OS also died with those OSs. Why is it next to impossible to open a file for exclusive access in Un*x without some application specific overhead ? Many other OSs handled this elegantly. The list goes on and on.



    Opinions are like rectums ! Everybody has one !!
      My Computer


  10. Posts : 2,497
    Windows 7 Pro 64 bit
       #40

    I stand by my statements 100%.

    When a language like Java or Python is involved it takes over some of the responsibilities from the application. But to the OS it is all one and the same. The OS has no knowledge and doesn't care whether the application is written in C, Python, Ruby, or assembly language. The application, language combination must obey the rules established by the OS.

    When a request is made of the OS (whether it comes form the application or the language makes no difference) that it cannot fulfill it has no reasonable alternative than to refuse the request. How an application or language handles this problem is up to it. Some handle this problem gracefully, others do not.

    I have been programming a variety of systems since the 1970's so have some knowledge of these matters. I expect that some statements on both sides have been misunderstood. I do not wish this to become an argument so I probably won't be making any further replies to this thread.
      My Computer


 
Page 4 of 6 FirstFirst ... 23456 LastLast

  Related Discussions
Our Sites
Site Links
About Us
Windows 7 Forums is an independent web site and has not been authorized, sponsored, or otherwise approved by Microsoft Corporation. "Windows 7" and related materials are trademarks of Microsoft Corp.

© Designer Media Ltd
All times are GMT -5. The time now is 23:53.
Find Us