Re: Samba read speed performance tuning
- From: Gary Gatten <Ggatten@xxxxxxxxxxx>
- Date: Fri, 19 Mar 2010 20:49:34 -0500
It MAY make a big diff, but make sure during your tests you use unique files or flush the cache or you'll me testing cache speed and not disk speed.
----- Original Message -----
From: owner-freebsd-questions@xxxxxxxxxxx <owner-freebsd-questions@xxxxxxxxxxx>
To: freebsd-net@xxxxxxxxxxx <freebsd-net@xxxxxxxxxxx>; freebsd-questions@xxxxxxxxxxx <freebsd-questions@xxxxxxxxxxx>; FreeBSD-STABLE Mailing List <freebsd-stable@xxxxxxxxxxx>; freebsd-performance@xxxxxxxxxxx <freebsd-performance@xxxxxxxxxxx>
Sent: Fri Mar 19 20:28:02 2010
Subject: Re: Samba read speed performance tuning
On Fri, Mar 19, 2010 at 11:14 PM, Dan Naumov <dan.naumov@xxxxxxxxx> wrote:
On a FreeBSD 8.0-RELEASE/amd64 system with a Supermicro X7SPA-H board
using an Intel gigabit nic with the em driver, running on top of a ZFS
mirror, I was seeing a strange issue. Local reads and writes to the
pool easily saturate the disks with roughly 75mb/s throughput, which
is roughly the best these drives can do. However, working with Samba,
writes to a share could easily pull off 75mb/s and saturate the disks,
but reads off a share were resulting in rather pathetic 18mb/s
throughput.
I found a threadon the FreeBSD forums
(http://forums.freebsd.org/showthread.php?t=9187) and followed the
suggested advice. I rebuilt Samba with AIO support, kldloaded the aio
module and made the following changes to my smb.conf
From:
socket options=TCP_NODELAY
To:
socket options=SO_RCVBUF=131072 SO_SNDBUF=131072 TCP_NODELAY
min receivefile size=16384
use sendfile=true
aio read size = 16384
aio write size = 16384
aio write behind = true
dns proxy = no[/CODE]
This showed a very welcome improvement in read speed, I went from
18mb/s to 48mb/s. The write speed remained unchanged and was still
saturating the disks. Now I tried the suggested sysctl tunables:
atombsd# sysctl net.inet.tcp.delayed_ack=0
net.inet.tcp.delayed_ack: 1 -> 0
atombsd# sysctl net.inet.tcp.path_mtu_discovery=0
net.inet.tcp.path_mtu_discovery: 1 -> 0
atombsd# sysctl net.inet.tcp.recvbuf_inc=524288
net.inet.tcp.recvbuf_inc: 16384 -> 524288
atombsd# sysctl net.inet.tcp.recvbuf_max=16777216
net.inet.tcp.recvbuf_max: 262144 -> 16777216
atombsd# sysctl net.inet.tcp.sendbuf_inc=524288
net.inet.tcp.sendbuf_inc: 8192 -> 524288
atombsd# sysctl net.inet.tcp.sendbuf_max=16777216
net.inet.tcp.sendbuf_max: 262144 -> 16777216
atombsd# sysctl net.inet.tcp.sendspace=65536
net.inet.tcp.sendspace: 32768 -> 65536
atombsd# sysctl net.inet.udp.maxdgram=57344
net.inet.udp.maxdgram: 9216 -> 57344
atombsd# sysctl net.inet.udp.recvspace=65536
net.inet.udp.recvspace: 42080 -> 65536
atombsd# sysctl net.local.stream.recvspace=65536
net.local.stream.recvspace: 8192 -> 65536
atombsd# sysctl net.local.stream.sendspace=65536
net.local.stream.sendspace: 8192 -> 65536
This improved the read speeds a further tiny bit, now I went from
48mb/s to 54mb/s. This is it however, I can't figure out how to
increase Samba read speed any further. Any ideas?
Oh my god... Why did noone tell me how much of an enormous performance
boost vfs.zfs.prefetch_disable=0 (aka actually enabling prefetch) is.
My local reads off the mirror pool jumped from 75mb/s to 96mb/s (ie.
they are now nearly 25% faster than reading off an individual disk)
and reads off a Samba share skyrocketed from 50mb/s to 90mb/s.
By default, FreeBSD sets vfs.zfs.prefetch_disable to 1 on any i386
systems and on any amd64 systems with less than 4GB of avaiable
memory. My system is amd64 with 4gb ram, but integrated video eats
some of that, so the autotuning disabled the prefetch. I had read up
on it and a fair amount of people seemed to have performance issues
caused by having prefetch enabled and get better results with it
turned off, in my case however, it seems that enabling it gave a
really solid boost to performance.
- Sincerely
Dan Naumov
_______________________________________________
freebsd-questions@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscribe@xxxxxxxxxxx"
<font size="1">
<div style='border:none;border-bottom:double windowtext 2.25pt;padding:0in 0in 1.0pt 0in'>
</div>
"This email is intended to be reviewed by only the intended recipient
and may contain information that is privileged and/or confidential.
If you are not the intended recipient, you are hereby notified that
any review, use, dissemination, disclosure or copying of this email
and its attachments, if any, is strictly prohibited. If you have
received this email in error, please immediately notify the sender by
return email and delete this email from your system."
</font>
_______________________________________________
freebsd-net@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscribe@xxxxxxxxxxx"
- Prev by Date: Re: kern/144874: [if_bridge] [patch] if_bridge frees mbuf after pfil hooks returns non-zero
- Next by Date: Re: Samba read speed performance tuning
- Previous by thread: Re: Samba read speed performance tuning
- Next by thread: Re: kern/144874: [if_bridge] [patch] if_bridge frees mbuf after pfil hooks returns non-zero
- Index(es):
Relevant Pages
|
