Re: GCC on multiprocessor platform?

_firstname__at_lr_dot_los-gatos_dot_ca.us
Date: 06/04/05

  • Next message: Greg Hennessy: "Re: Networking with OpenBSD and Linux/Windows" 
    Date: Sat, 04 Jun 2005 05:08:40 -0000

    In article <d7r1lc$7d5$1@news.onet.pl>, m <sq8ijk@poczta.onet.pl> wrote:
    >Marco S Hyman napisał(a):
    >> m <sq8ijk@poczta.onet.pl> writes:
    >>>How to use more than one cpu (I am using bsd.mp) for building something
    >>>e.g. make
    >>
    >> Make sure that there are enough processes running to require all available
    >> cpus. On my two processor system, for example, it takes something
    >> like 8 concurrent compiles to keep both processors busy most of the time.
    >>
    >you where right, I used before one more process than cpu (like I read
    >somewhere in gentoo faq) but now I set up more and CPU usage grows up :)

    We tested this extensively under HP-UX once. We found that the optimum was
    8 compilation processes per CPU (we only tested powers of two, and above 4
    processes per CPU there was little difference). Meaning on an 8-way
    machine, you had to use -j64, and there are very few source trees where this
    can be put to full use. The answer depends somewhat on the IO system, the
    total amount of memory, and the ratio of memory bandwidth to CPU speed. It
    makes a huge difference if the complete source tree is already in cache when
    you start the compilation (for example it was just untared); then the
    bottleneck is not the incoming IO system, and you need fewer compilation
    processes to saturate the CPU. A lot of memory (many gigs) and a large TLB
    is a must for this kind of game. The answer on an x86 with IDE disks will
    probably be somewhat different from the answer for a high-end RISC machine
    with fibre channel and a high-end RAID subsystem. But -j5 on a 4-way
    machine is way too low. 

    -- 
The address in the header is invalid for obvious reasons. Please
reconstruct the address from the information below (look for _).
Ralph Becker-Szendy      _firstname_@lr_dot_los-gatos_dot_ca.us
  • Next message: Greg Hennessy: "Re: Networking with OpenBSD and Linux/Windows"

    Relevant Pages

    • Re: What is a type?
      ... >> compilation time. ... The type int is perfectly supported by ... You probably won't find the term "int" in a CPU reference manual. ... I meant that a function pointer could be implemented as an index into ...
      (comp.lang.c)
    • Re: GCC on multiprocessor platform?
      ... > processes per CPU there was little difference). ... > total amount of memory, and the ratio of memory bandwidth to CPU speed. ... > bottleneck is not the incoming IO system, and you need fewer compilation ... -j3 on a two-way box was plenty fast enough to build a Linux ...
      (comp.unix.bsd.openbsd.misc)
    • Re: dual processor machine
      ... I don't think CPU matters ... > Compilation will be I/O limitted? ... lots of small source files is certainly more I/O bound than compiling a ... few large source files. ...
      (comp.os.linux.hardware)
    • Re: [PATCH [RT] 08/14] add a loop counter based timeout mechanism
      ... into Kconfig. ... Compilation is usually the wrong place to configure ... It looks suspiciously round and worse the actual spin time depends a lot on the ... differently from a 2Ghz CPU) Did you experiment with other spin times? ...
      (Linux-Kernel)
    • fun with pools: how to create a compilation script that spreads itself on multiple cpus
      ... Could someone pass a straw on how to paralellize compilation on multiple ... Is it safe to enable both static and dynamic pools or I should choose ... pass each form per CPU ...
      (SunManagers)