Re: How analyze the system bottleneck using shell tools

Bo Yang <struggleyb@xxxxxxxxx> writes:

Well, normally the memory will fill up with a cache of the file
system. This isn't happening in your case. If the system was
rebooted recently, I'd expect numbers like this. If it's been up for
days, i'd expect it to be more used and less free. If it is always
free, then I think this is very strange. All of the disk would be
loaded into memory, and your disk I/O would drop.

I don't understand why the free memory is so high after the system has
started for a month.


I'm not sure either. Perhaps if everytime an entry is modified, it's
written to disk (the database), the memory is freed.
As the other poster suggested, perhaps your system is I/O limited.

So, 5000 means context switch 5000 times per second. And I think that
means that my system application are all not CPU-intensive
application. They need network and I/O more.

I think so.

And I don't understand what do you mean by saying that the average for
my computer. I just want to know, for my computer, what cs is the
normal based on my hardware. You mean the benchmark tool can give the
answer, right? Could you please explain more for me? Thanks!


Each system has limits in I/O, CPU, disk speeds, etc. The benchmark
programs will help you find what the maximum limit it.

If, for instance, your system has a maximum limit of 10,000 context
switches a second, and your system is at 5000 - you are not near the
limit.

On the other hand if the benchmark shows your maximum limit is 5500,
then 5000 is about as fast as it can go, and is the indication of the
problem.

The trouble is - a good benchmark is done when the system is
completely idle. If your server is busy all the time, then the
benchmark will slow down the experience to the users.

If you can run it as a time when the normal load is small, and the
benchmark only takes a few seconds, you could try that. If the
benchmark takes an hour, that might be unacceptable to your users.



You mean my machine use RAID. I am a new comer for this machine and I
am not familiar with RAID. too.

A RAID is a Redundant Array of Inexpensive Disks.
The disk controller needs to be RAID compatible.

I'm not an expert, but there are different types of RAID systems. One
way to configure it is to do disk striping. Instead of putting a
partition on a single disk, you put it on two disks - making it one
virtual disk. If one disk as a limit of 5000 blocks per second, by
using 2 disks in parallel, you can increase this to twice that limit.

The down side is that if you lose either disk, the partiton fails.


A RAID is one way to speed up I/O. As I said, you can also get a
faster disk, or use a second disk controller. You can also split up your
I/O onto different disks.

Beside above, I have a additional question. Could you please explain
how Linux measure the CPU time. I mean what will be count as sys time
and what will be count as wa time? thanks!


A program runs in user space normally. A program that just did
calculations would be 100% in user space.


When a program needs he resources of the system, it places a call
using one of the functions listed in section 2 of the manual pages.
In other works, the user asks the system to do something. Examples include
Execute a program
Do disk I/O
Network calls
Communicate to a device driver (Terminal, display, printer, mouse)

Sometimes the user makes a request and the system has to do something
to fulfill that request. If the program needs more memory, the user
program is halted, and the system pages in the virtual memory, and the
program continues.

Many times, when the user requests disk or network resources, the
request is initiated, and the user program sleeps until the I/O is
finished.

There is also a clock that runs and gives other programs a chance to
run. So perhaps 1000 times a second, the system wakes up. If a
program is waiting for the disk, the system wakes up the program and
lets it get the results.

So the user time versus system time indicates if the CPU is busy doing
something for the user, or for the OS. I don't know what is normal,
but if the system time is high, and the user time is low, that's
indication of the system struggling. Usually the user time is high.

In older unix systems, the total time was user + system + idle. Your
system also has other values:

avg-cpu: %user %nice %system %iowait %steal %idle
4.13 1.26 11.02 7.70 0.00 75.89

Nice is a term for low priority tasks. If you have a task that you
want done, but don't want it to interfeer with other programs, you
"nice" it.

I don't know what "steal" is.

You have 75% idle time.
You have more system time that user time.
The user time is only 4%. If this is when the performance is slow, that's bad.

This would indicate that the OS is trying to get the jobs done, but it
has to do a lot of internal bookkeeping. It's struggling (more system than user).
But it's not CPU bound, because you have 75% idle.
The 7.7% IOWAIT suggests the problem is disk related.
The other posted indicated this, and he has more recent knowledge than I.

I think that's your problem.
.

Relevant Pages

  • Filegroups and multi-processor threading improvements
    ... I have read extensively on filegroups vs. RAID 5, ... threads that SQL 2000 will use per filegroup. ... I/O will be limited to one CPU thread. ... Granted disk I/O ...
    (microsoft.public.sqlserver.server)
  • Re: file system (UFS2) consistancy after -current crash? (fwd)
    ... The problem that we have been having with many of the RAID ... systems is that they give an I/O completion interrupt after ... they copy the change into their memeory, but before the I/O ... disk, it proceeds to the next step. ...
    (freebsd-current)
  • I/O request taking longer than 15 seconds
    ... I/O request taking longer than 15 seconds to complete ... I've seen this same problem and my suspicions are that the disk defrag is ... reading or writing to your data or log files? ... getting your feedback but unfortunately that didn't resolve the problem ...
    (microsoft.public.sqlserver.server)
  • I/O request taking longer than 15 seconds
    ... I/O request taking longer than 15 seconds to complete ... I've seen this same problem and my suspicions are that the disk defrag is ... reading or writing to your data or log files? ... getting your feedback but unfortunately that didn't resolve the problem ...
    (microsoft.public.sqlserver.server)
  • Re: Caching control
    ... |> | invalidate/unmap them in order to discard the data from memory. ... |> writing out to disk. ... | easy to discard as clean disk cache. ... stating that a specific amount of RAM can be used only for I/O ...
    (comp.os.linux.development.system)