Re: what to do with "too much" RAM?

From: Andrew Gabriel (andrew_at_cucumber.demon.co.uk)
Date: 07/03/05 

Date: 03 Jul 2005 20:38:30 GMT

In article <da9dod$9bk$1@eol.dd.chalmers.se>,
        dol@ce.chalmers.se (Fredrik Lundholm) writes:
> In article <da990p$is9$1@news.urz.uni-heidelberg.de>,
> Dennis Grevenstein <dennis.grevenstein@gmail.com> wrote:
>
>>Sure, I could put the whole system into a RAM disk,
>>but is there any way to just improve write-caching?
>
> Absolutely not. Well there are file system tunables
> and such but the general rule is to NEVER cache writes.

You're in for a big disappointment -- all Unixs normally
cache writes, although it depends on the nature of the
data being written.

> Remember why you make them in the first place.
>
> You want writes to fly out to the raid controller as fast as possible
> and do the caching there.

That depends what the raid controller and disk controllers
are going to do with the data, but it isn't necessarily any
safer there than in main memory. Indeed it can cause very
bad problems if the writes get reordered when going to
non-volatile storage outside the control of the OS.

> Why dont you want to cache writes at the host?
> Think about server crash, power loss, file system integrity and do the math.

When applications require data to be secured to non-volatile
storage, they will ask for the data to be sync'ed to disk
(or non-volatile cache). Otherwise, write-behind cacheing is
the norm. (Some specialist applications may turn off the
cacheing of their data by the OS, generally because they do
it themselves and have a better idea of what's good to cache
and what isn't.) 

-- 
Andrew Gabriel

Relevant Pages

  • Temporary lock-up under heavy write, MegaRAID RAID-5
    ... LSI Logic MegaRAID 320-1, 64MB cache ... when writing very large files. ... disk write access, actually). ... The RAID controller is currently set to "write-through". ...
    (Debian-User)
  • Re: process memory and page cache clarification is needed
    ... Can we say that For directio (this is what i really use, ... If i use directio, instead of using page cache, the data is directly ... is necessary it can be paged out to swap disk. ... inside the page cahce which are dirty and hold file system data. ...
    (comp.unix.solaris)
  • Re: Fast file writes without having large data flushes to disk
    ... file system driver, I assumed that it's a separate component, and the lower ... level drivers have other cache mechanisms, ... just disk activity; this is most likely an OS or disk driver ... And are there tools that can show the amount of file system ...
    (microsoft.public.win32.programmer.kernel)
  • Re: Google SoC idea
    ... In my opinion the original proposal described a non volatile write cache ... The proposed write cache implementation uses journaling. ... This does not offer additional guarantees to a file system and should ... (Mirrored 15K SCSI write cache for big, fat slow RAID 5/6 SATA disk ...
    (freebsd-hackers)
  • Re: Caching control
    ... |> | invalidate/unmap them in order to discard the data from memory. ... |> writing out to disk. ... | easy to discard as clean disk cache. ... stating that a specific amount of RAM can be used only for I/O ...
    (comp.os.linux.development.system)