32bit 4Gb Memory Limit Explained

[Corpsecrank]

I found an interesting article that I thought was worth sharing. I have always been bugged when reading that 32bit operating systems were limited to 4Gb of memory because of the way memory is addressed. It never made any sense to me because when I took Windows XP classes it had a nice little table covering different versions of Windows in the back which showed certain 32bit versions allowing up to 128Gb of memory to be recognized.

The book was written for taking Microsoft exams to earn a certification in XP and I also had another book of the same type for Server 03 again with the same table in the back. So if Microsoft is saying that 32bit is not limited to 4Gb why are people trying to say that the limit is because 32bit is not capable of addressing it?

Because the standard 32bit home desktop versions of Windows are not licensed to allow more than 4Gb. The only reason you are limited is because Microsoft wrote the OS to limit you not because 32bit cannot address memory above 4Gb.

If you want to read more about this in full detail with some images where the guy actually removed the limitation by altering the license file you can do so here:

Geoff Chappell, Software Analyst - Viewer

 

[Dinesh]

Nice find +rep. BTW, the hack is very ugly.

 

[Corpsecrank]

Nice find +rep. BTW, the hack is very ugly.

Thanks Yeah I love it when he says PAE is an ugly hack lol.

I think this article is great because it finally answers one of the biggest most misunderstood questions about 32bit windows and memory addressing. The only problem is that now since I understand it when I run into articles where people do not it is even more irritating to read all the misleading information they believe is correct.

One guy even tried to say that 32bit processors were the cause and that the reason PAE could allow more memory to be addressed is because 32bit processors were actually designed as 36bit and limited to 32bit however the entire thing is easily dispelled because he failed to explain why a 64bit processor still had the same limitations.

 

[Dinesh]

I liked the way he has honestly written this article. Microsoft has been lying all these times.

 

[Corpsecrank]

I liked the way he has honestly written this article. Microsoft has been lying all these times.

Well not so much lying they just didn't mention the way memory was handled.

It has been my experience that any type of limitation set by Microsoft is nicely labeled as a license so that it sounds better.

The problem with this case is that they never mentioned that memory usage was something they licensed.

 

[QuackPot]

Good read OP. Must be a marketing ploy or rake in more revenue.

+rep (I gave you another pip in your rep )

 

[logicearth]

The 4 GB limit for 32 bit is a technical limitation. There is just not enough addressing space in 32 bit integer lengths. To get around this issue you must increase the bit length, either by using 64 bit or with PAE. Now PAE must be supported not only by the hardware but also the OS and its core services (aka drivers). So why does Microsoft limit the 32 bit OS to 4 GBs of RAM? Well simple really, most of the hardware and drivers out there bulk with more then 4 GBs of RAM.

When you have an OS installed on billions of computers each with a unique set of hardware, why take the risky path? Limit to 4 GBs is the wisest choice to make. And honestly, Microsoft doesn't have to tell us jack about it (Microsoft artificially limits RAM all the time, Home Premium 64 bit is limited to 16 GB). That does not make it a marketing ploy or to rake in more revenue. Jeeze the 64 bit version cost the same as the 32 bit and they even come in the same box! Or you can get the 64 bit if you need it and use your 32 bit CD key.

Wild and silly speculation...meh.

 

[whs]

An interesting article indeed. The way I read it is that 32bit and 64bit OS are essentially the same code and both are present when you buy 32bit. With some hacks and tweaks you can disable the 32bit blockage and make it run as 64bit. That's like taking the muffler out of the exhaust pipe of a mofa in order to make it run faster - and that is not legal either.

 

[Corpsecrank]

The need for more than 4Gb of ram in a 32bit environment for a home user no matter what type of applications are being used is non existent really. Which is my guess why allowing more than 4Gb as a standard was never an option from the start.

The only time you really need more than 4Gb is if your running a server and if that were the case you would want to run a server version of windows which would support up to 128Gb depending on the version used.

I have done some pretty heavy operation with my current setup and never came close to touching the 4Gb of installed ram. I always have around 1200Mb of free ram even when running multiple apps that require a large amount of ram.

The reason it is so hard to fill the ram up though is mainly due to the speed at which it can pass off data.

 

[H2SO4]

The 4 GB limit for 32 bit is a technical limitation. There is just not enough addressing space in 32 bit integer lengths. To get around this issue you must increase the bit length, either by using 64 bit or with PAE. Now PAE must be supported not only by the hardware but also the OS and its core services (aka drivers). So why does Microsoft limit the 32 bit OS to 4 GBs of RAM? Well simple really, most of the hardware and drivers out there bulk with more then 4 GBs of RAM.
...
Wild and silly speculation...meh.

Completely agree. That article is entirely misguided. I think the conspiratorial tone was meant to raise his profile, as opposed to actually explaining the issue properly. A few other points...

The PAE requirement for 36-bit addressing was/is not met by all consumer-level processors. You need the motherboard and processor to provide a 36-bit address bus before anything else can happen.

"Client" Windows versions used to have this functionality. It was deactivated in XP SP2 and all subsequent versions because it was inducing BSODs in buggy drivers which did not deal well with the remapping. The dilemma faced by MS came down to two choices:

1) Work with every driver provider in the desktop space to teach them how to properly deal with PAE-remapped memory and hence not BSOD machines and corrupt data.

2) Deactivate the remapping for non-server versions of Windows since most 32-bit home/small-office users don't tend to sport in excess of 4GB RAM anyway.
​

The first one is obviously somewhat utopian, so they chose the latter - no remapping on "client" Windows as of XPSP2 and later. If you want to run more than 4GB, go with 64-bit.

There's no conspiracy. The guy's a ranter.

 

[Dinesh]

Now i m confused whether H2SO4 is right or the OP.

 

[zigzag3143]

I think H2SO4 and the OP are saying similar things. OP says its possible in hardware and it is but its microsofts fault. Sulphuric is saying its possible but here is why it wasnt done and shouldn't be done.

Im sure if I have mis represented either they will let me know. Im no expert

Ken

 

[logicearth]

Now i m confused whether H2SO4 is right or the OP.

Simple really if you want lots of RAM and huge performance 64 bit. Not only do you get larger address tables, but the processor can handle bigger numbers at one time versus 32 bit.

 

[jimbo45]

Hi there
This issue is slightly more complicated than would seem at first sight.
From a purely Mathematical standpoint 4GB is the maximum address that a 32 bit system can address -- 2 **32.

However some OS'es can "poodle fake" the addressing scheme to allow a higher range than this -- it also needs some extra hardware BTW.

What Server 2003 R2 (32 bit system) does for example (it can address up to 64GB RAM) is to arrange the addressing scheme as follows.

Instead of using a 32 bit address it takes the First 4 bits and uses these like a telephone area code for example 213 / 450 etc etc.

This means that the other 28 bits can be used as an address but you can have up to 16 banks of these.

So your address ranges could be something like this
1FFF FFFF
2FFF FFFF etc.

Note the OS must have methods of saving and "switching" the prefix byte to correctly address the the correct range of memory -- and it needs to be FAST otherwise the OS suffers.

I believe the hardware feature is called something like PAE and it must be enabled in the CPU for this to work -- I'm not sure of the exact name but that's conceptually how it works.

This feature isn't really of any use in 32 bit desktop os'es but was implemented for servers --when you have large Databases / servers with lots of concurrent users you DO need a load more RAM -- and at that time 64 bit CPU's weren't generally available so the method outlined above was a bit of a workaround.

Cheers
jimbo

 

[H2SO4]

Now i m confused whether H2SO4 is right or the OP.

Simple really if you want lots of RAM and huge performance 64 bit. Not only do you get larger address tables, but the processor can handle bigger numbers at one time versus 32 bit.

What logicearth said. That's the crux of the matter.

I think H2SO4 and the OP are saying similar things. OP says its possible in hardware and it is but its microsofts fault. Sulphuric is saying its possible but here is why it wasnt done and shouldn't be done.

Im sure if I have mis represented either they will let me know. Im no expert
Ken

My disagreement is primarily with the author of the referenced article. His conclusions are painted in such a way as to encourage people to feel ripped off, even though I suspect he probably has sufficient technical knowledge to understand that he's being deliberately biased. Longer version:

No pure 32-bit system can EVER go beyond 4GB (using the flat memory model). There are insufficient bits to describe any bigger numbers.

In the mid-90s, Intel recognised a looming problem and released processors capable of 36-bit addressing. When plugged into a corresponding motherboard, such processors could theoretically address up to 2^36 or 64GB of memory. It required the OS and drivers to support a mechanism known as PAE remapping so that regions of memory above 4GB became visible and usable.

This is not a trivial thing to support from a driver's point of view. Many of the numerous consumer-grade drivers (NICs, audio, video, AV, whatever...) were fine in a pure 32-bit environment, but they exhibited severe problems when attempting to deal with the complexities of PAE remapping. These issues manifested themselves as BSODs or silent data corruption.

Eventually, MS came to the conclusion that the 99% of home/small business Windows users who had 4GB of RAM or less were paying a BSOD penalty for the other 1% to be able to enjoy their >4GB machines. Their response was to deactivate PAE remapping functionality in "client" versions of Windows (XP, Vista, Win7).

"Server" flavours are different because the drivers are generally written to a much higher standard, plus the onus is on the (expert) system administrator to be judicious in their driver selection. Hence, 32(36)-bit "Server" Windows builds can still utilise PAE remapping to go beyond 4GB and all the way up to 64GB for the "Enterprise" and "Datacenter" versions.

 

[pallesenw]

Before you guys write more about x86 architecture, PAE and driver development in Windows, why don't you try and research it first? Not much of what has been said here is true.

 

[logicearth]

Before you guys write more about x86 architecture, PAE and driver development in Windows, why don't you try and research it first? Not much of what has been said here is true.

So what is not true? Care to enlighten us?

 

[pallesenw]

The driver issue: as long as you write your driver according to the rules (and proper API), you don't need to care about what mode you might or might not run in, and how much ram there is installed. Some driver developers just think they are smarter than Windows and know every possible hardware configuration out there (which they don't and therefore might crash).

You said something about integer and register size. That is not relevant. PAE is about physical addressing only, not virtual.

 

[H2SO4]

The driver issue: as long as you write your driver according to the rules (and proper API), you don't need to care about what mode you might or might not run in, and how much ram there is installed. Some driver developers just think they are smarter than Windows and know every possible hardware configuration out there (which they don't and therefore might crash).

I think it's a tautology to say that perfect software behaves perfectly.

You said something about integer and register size. That is not relevant. PAE is about physical addressing only, not virtual.

Surely you didn't join this fine forum just now only to point out that the 'P' in PAE stands for "physical"? If you believe there are technical inaccuracies in the information provided, please consider expanding on your points and providing details so that everyone can learn.

 

[Corpsecrank]

Before you guys write more about x86 architecture, PAE and driver development in Windows, why don't you try and research it first? Not much of what has been said here is true.

So what is not true? Care to enlighten us?

Yeah good call because I think most of us are in agreement in this thread so far.

Most of what has been said is in agreement with posts before it and simply adds more facts to the discussion. And what has been said so far is in fact truth not just blabber from people that have not done homework on this.

Where did the original topic come from? Research into the subject how else would this all have been brought about.

Anyways so far those who have contributed to the discussion have all pretty much been stating the same thing except with added bits and pieces that help fill this all in nicely.

I think there is a lot to be learned reading this thread so far.

As for the author of the article? Well I won't say I completely agree with the view he has because he makes microsoft out to be a bad guy when really they never sat there and conjured up some way to limit ram so that we would have to pay more money to get more performance. I think that over the years developing the OS things just kind of fell in place to be what they are today.

But this thread is only intended as a learning experience because there is so much misleading information floating around the net related to PAE (Physical Address Extension) and memory mapping in general. I figured perhaps we could all gain a bit of knowledge from this. I think we have accomplished just that.