Re: amrd disk performance drop after running under high load



Alexey Popov wrote:
Hi.

Kris Kennaway wrote:

After some time of running under high load disk performance become expremely poor. At that periods 'systat -vm 1' shows something like this:
What does "high load" mean? You need to explain the system workload more.
This web service is similiar to YouTube. This server is video store. I
have around 200G of *.flv (flash video) files on the server.

I run lighttpd as a web server. Disk load is usually around 50%, network
output 100Mbit/s, 100 simultaneous connections. CPU is mostly idle.

As you can see it is a trivial service - sending files to network via HTTP.

A couple of comments.

Does lighttpd actually use HTTP accept filters?

Are you using ipfilter and ipfw? You are paying a performance penalty for having them.

You might try increasing BUCKET_MAX in sys/vm/uma_core.c. I don't really understand the code here, but you seem to be hitting a threshold behaviour where you are constantly running out of space in the per CPU caches.

This can happen if your workload is unbalanced between the CPUs and you are always allocating on one but freeing on another, but I wouldn't expect it should happen on your workload. Maybe it can also happen if your turnover is high enough. What does vmstat -z show during the good and bad times?

Kris


_______________________________________________
freebsd-hackers@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@xxxxxxxxxxx"



Relevant Pages

  • Re: amrd disk performance drop after running under high load
    ... What does "high load" mean? ... This server is video store. ... I don't really understand the code here, but you seem to be hitting a threshold behaviour where you are constantly running out of space in the per CPU caches. ... This can happen if your workload is unbalanced between the CPUs and you are always allocating on one but freeing on another, but I wouldn't expect it should happen on your workload. ...
    (freebsd-stable)
  • Re: amrd disk performance drop after running under high load
    ... This server is video store. ... Disk load is usually around 50%, ... amrd disk performance drop after running under high load ...
    (freebsd-hackers)
  • Re: amrd disk performance drop after running under high load
    ... This server is video store. ... Disk load is usually around 50%, ... amrd disk performance drop after running under high load ...
    (freebsd-stable)
  • Is this load reasonable?
    ... I'm getting a rather high load but my CPU is mostly idle and my IOwait is ... r b w swpd free buff cache si so bi bo in cs us sy ...
    (linux.redhat)
  • Re: CIFS/SMBFS failing under load in 2.6.X
    ... Well the server is under very high load in this test and the ... in 2.4.X does not seem to timeout. ... SMBFS only 4K. ...
    (Linux-Kernel)