Re: Memory management/File IO related problem.

From: Darren Dunham (ddunham_at_redwood.taos.com)
Date: 09/09/03

  • Next message: Darren Dunham: "Re: High density on MSL5026 Tape Library" 
    Date: Tue, 09 Sep 2003 04:30:54 GMT

    Nikos Massios <massios@inl.nl> wrote:
    > We are using:
    > Solaris 7, gcc 2.8.1

    > Our application:
    > A process runs consecutively with several different
    > parameters. Each time, depending on the parameters, different
    > (small) parts of a large binary file are accessed with
    > fseek, fread. To give some idea on the sizes involved
    > the file is around 900mb and the total data read each time
    > is around 1mb. This 1mb is distributed at around 1000
    > different places in the 900mb file. At each place roughly
    > the same amount of data is read (around 1kb).

    > Our problem:
    > It seems to us that Solaris 7 is mapping parts of the file
    > to memory automatically (we can also map the entire file
    > with mmap if we want to). It is to be seen (1) while each
    > process is running and (2) after each process has completed.

    That seems incorrect. Can you do a 'pmap -x PID' for the process both
    before and after the memory increase? Are there new segments mapped, or
    are existing segments larger?

    Simple 'fread' stuff shouldn't add anything to the process size. The
    file will be cached, but it will be cached in space managed by the
    kernel and the filesystem, not in the process space.

    My first assumption is that something else is going on in your code, but
    it's difficult to be certain.

    > The Question:
    > Is there a way to either:
    > a) Disable completely the memory mapping of files used by
    > our processes. Either in C or in Solaris.

    It shouldn't happen at all in the way you describe. 

    -- 
Darren Dunham                                           ddunham@taos.com
Unix System Administrator                    Taos - The SysAdmin Company
Got some Dr Pepper?                           San Francisco, CA bay area
         < This line left intentionally blank to confuse you. >
  • Next message: Darren Dunham: "Re: High density on MSL5026 Tape Library"

    Relevant Pages

    • Memory management/File IO related problem.
      ... It seems to us that Solaris 7 is mapping parts of the file ... Initially the memory used by a single process is small ... Also, after the process has completed, some free memory is ... Disable completely the memory mapping of files used by ...
      (comp.unix.solaris)
    • Re: Canadian OpenVMS Seminar (07.02.20)
      ... appropriate answer to the question of "What specific RAS-related features ... Re: Solaris x86: Neil wrote about "running Solaris or LINUX on x86 to see ... support for DDR2 memory and support for online spare memory chips, ... relationship with Intel)? ...
      (comp.os.vms)
    • Re: steps to handle system problems
      ... User reports of "The application is slow" is just noise unless you can pin down time of day, activities on the network or server running it. ... I/O and memory stats over the long term to look at reports of "slow" with time and activity. ... In a galaxy long ago and far far away, an instructor in system tuning told my class that you get the most bang for your buck by first tuning the application, then maybe 10% from tuning the system. ... She was talking about another platform and O/S entirely different from Sun Solaris. ...
      (comp.unix.solaris)
    • Re: Decoding strategy
      ... may be awkward to replace memory mapping with FileStream and measure ... the OS simply cannot correct predict what to buffer for you. ... just reading normally. ...
      (microsoft.public.dotnet.languages.csharp)
    • Re: UNEXPECTED SOFT UPDATE INCONSISTENCY; RUN fsck MANUALLY
      ... I'm curious to know how Solaris deals with these problems, ... default filesystem in OpenSolaris is now ZFS. ... The ARC is separate from the general VM cache in solaris, too, IIRC. ... Most of the problems where ZFS runs the machine out of kernel memory (or ...
      (freebsd-stable)