Re: CPU waiting for... what? (mistery)

From: Darren Dunham (ddunham@redwood.taos.com)
Date: 04/12/03 

From: Darren Dunham <ddunham@redwood.taos.com>
Date: Fri, 11 Apr 2003 23:19:25 GMT

In comp.unix.solaris Rick Denoire <100.17706@germanynet.de> wrote:
> Darren Dunham <ddunham@redwood.taos.com> wrote:

>>CPU wait states say almost nothing about I/O. All it really says is
>>that the CPU is somewhat idle. I/Owait + idle equals true CPU idle. So
>>a box with 70% io wait has lots of CPU for processing.

> Disagree. I/O wait: CPU would do something, just can't.
> CPU idle: there is really nothing to do.

Yes, but the difference has almost nothing to do with how much I/O is
occuring.

I can take a 8 CPU box, run a single mt command to access a tape, and I
will have an 8 CPU machine showing 99% I/O wait. All that means is that
the CPUs are idle.

I/O wait indicates there is at least one outstanding I/O. If your
application is more I/O intensive than CPU intensive, your system will
probably accumulate some iowait time. You cannot use that figure to
determine if the system is healthy or unhealthy.

> Doing nothing has different causes.

>>"wait queue"? What figures are you using that show you that? Are you
>>talking about the "b" column from vmstat?

> Sorry. I meant the secound column you get with vmstat. This is the
> number of blocked processes.

Gotcha. Then my answer below stands. There's no utility within Solaris
that will directly show you a process that is in the 'r', 'b', or 'w'
state.

>>How do you know you have an I/O bottleneck at all? You mention that
>>you've seen 45MB/s transfer rates. That sounds good. As long as iostat
>>is showing that the service times are small, then I think you've proved
>>that the I/O is okay.

> I am not that impressed by 45 MB/s if you consider that this is a
> Raid. Any top consumer type harddisk can achieve this sustained
> transfer rate ALONE.

> Actually this is a poor performance. The host should be reading from
> cache due to the advanced prefetch mechanism of the storage subsystem.
> Reading from cache is much faster.

Then start your search there. Ignore the iowait states. If the
performance of the storage is poor in isolation (dd, bonnie), then I'd
try to figure that out before moving on to the performance of the
application.

I'd even start below the filesystem if possible. Can you deliver
sequential data to the underlying devices with 'dd' at a good rate? Can
you do so with the filesystem in place?

>>Difficult. I think you'd have to write a utility that looked at the
>>status of all the processes on the machine and recorded if any of them
>>were in the "blocked" state. I have some information from someone else
>>about the general aspects, but I've never attempted to code it up.

> etruss was supposed to do that, it just does not work properly.

What kind of output are you getting from it? 

-- 
Darren Dunham                                           ddunham@taos.com
Unix System Administrator                    Taos - The SysAdmin Company
Got some Dr Pepper?                           San Francisco, CA bay area
         < This line left intentionally blank to confuse you. >

Relevant Pages

  • IO WAIT Information From IBM
    ... the I/O wait metric in AIX. ... AIX scheduler, the CPU "queues", the CPU states, and the idle or wait ... To summarize it in one sentence, 'iowait' is the percentage ...
    (AIX-L)
  • userland starvation with 2.4.25-rc2
    ... I am trying to load the SCSI disk by doing in parallel: ... The upper graph is the CPU load, the lower graph is the I/O load. ... you can see, at certain points the kernel will take all available CPU, ...
    (Linux-Kernel)
  • Re: IO WAIT Information From IBM
    ... >understanding of how the I/O wait value is collected and calculated. ... >of the CPU resource. ... The wait processes only job is to increment the counters that report ... >Each CPU can be in one of four states: user, sys, idle, iowait. ...
    (AIX-L)
  • Re: IO WAIT Information From IBM
    ... >understanding of how the I/O wait value is collected and calculated. ... >of the CPU resource. ... The wait processes only job is to increment the counters that report ... >Each CPU can be in one of four states: user, sys, idle, iowait. ...
    (AIX-L)
  • Re: It is almost certain now, INTEL will have 64bit x86 !!
    ... >>between this an 8400 which was that you had CPU boards, memory ... > if the MaxCPU boards had memory) number of CPUs. ... I/O bridge x 16! ... In the F15K these can be used for I/O or 2 CPU MAXCPU cards ...
    (comp.os.vms)