Re: 4GB VM for 32 bit processes

From: Manish Baronia (manishbaronia_at_hotmail.com)
Date: 08/28/03 

Date: Thu, 28 Aug 2003 10:54:34 +0530

Loic Domaigne wrote:

> Hi Manish,
>
> I'm not an expert, but I will attempt to answer some of your
> questions.
>
> > I have a question regarding the limit of resource (VM), a
> > process can cosume on 32bit unix (SunOS 5.7, Red Hat Linux 7.2,
> > AIX 4.3, HP-UX 11.00) machines.
>
> Obviously, max. 4GB ;-)
>
> > Let us assume that Physical+Swap memory on these systems is > 4GB.
> > Theoretically, can a process consume 4GB of VM (private + shared)
> > on all these machines, or is there any limit imposed by these
> > systems which can not be overcome even by changing kernel
> > parameters ?
>
> Certainly, the amount of VM available to a process is limited by the
> ulimit. But this can be eventually 'adjusted' on-the-fly.
>
> However, as explained by Casper, the 4G are often divided into two
> areas: user-space and kernel-space. For instance, on Linux, a 3G-1G
> splitting is common. Some Un*x systems go e.g. for a 2G-2G splitting.
>
> On Linux, this splitting can be configured. But it requires that you
> re-compile the kernel (this is, in a sense, a kernel parameter. But
> you can't change it "on the fly").
>
> > I am asking this question, because somewhere, I read that linux has a
> > limitation of 3GB, while HP 2GB. I assume, these limits are of
> > private memory. With shared memory (mmap) this should go till 4 GB.
> > Do I get it correct?
>
> Assuming a default config, even if theoritically 3G are available on
> Linux, you will probably be able to mmap only 2G approximatively (the
> process itself, of course, needs also VM for the text and data!). The
> reason is that the 3G are further divided into two areas: 1G for the
> heap, and 2G for "the rest". I don't know how these last splitting can
> be configured...
>
>
> What's about PAE, which can address 64G? Linux 2.6.0 (beta) offers the
> possibility to have a 4G-4G mapping for the user-space / kernel-space
> (again, you need to compile the kernel with that option enabled).
> However, I still believe that the 1G for the heap remains, so in that
> case you will be able to mmap() only 3G max.
>
> The workaround to use more VM on PAE, is to fork several instances of
> the process and use some IPC. That's definitively not so easy as with
> a linear addressing, but remember that we hit the 32 bits hard limit
> here. So we can't expect miracles...
>
> My answers are definitively Linux centric, but some of the material
> provided should also apply to other Un*x systems. Of course, on 64
> bits arch, all these problems vanish...
>
> BTW, a final comment. You can easily write a small program that does
> nothing but mmap chunks of memory (eg 1M) until it fails. That way,
> you know the amount of VM available for shared memory on your system.
> For instance, one my Linux box (standard 3G-1G splitting), I'm able to
> mmap 2,046 M (the 0.002 M are for text and data of the program
> itself).
>
> Regards,
> Loic.

Thanks Jerry, Casper, Loic, Eric,
        The answers do give me insight of memory management. Does
getrlimit(RLIMIT_AS, &rlimit) ; rlimit.rlim_max returns me correct user
area value(According to kernel parameters)?
Thamks,
Manish

Relevant Pages