Tweaking Disk Cache/Buffers and fsync

There's a lot of discussion on the sysctl settings for vfs disk
cache/buffers,

but very little consolidated or comprehensive explanations.



I have a NAS box with 1GB of memory, but rarely see over ~40MB of
utilization

during file transfers. When writing a large file to the NAS, I see the
transfer

rate throttle and pause in regular intervals. This indicates to me the dirty

buffers are flushing and writing to the disk, and the file transfer is
interrupted

during the disk write process.



Therefore, I would like to (1) increase the memory utilization, so that a
large

file write (500MB) will buffer entirely to memory for fastest transfer
speeds,

and (2) then take as long as needed to write the dirty buffers to the disk,
after

the transfer is concluded.



For part (1), I looked at the following sysctl settings, but I don't fully

understand the interactions:



vfs.maxbufspace

vfs.lobufspace

vfs.hibufspace

vfs.hidirtybuffers

vfs.hirunningspace



also /boot/loader.conf kern.nbuf



I understand some rules: hirunningspace should be between 1MB to 4MB.

lobufspace should be 25% to 75% of hibufspace.



What I don't understand is the relationship between maxbufspace and
hibufspace,

and what are good values.



I also don't understand hidirtybuffers.



For part (2), I need to slow down the occurrence of sync() or fsync().
Ideally,

a sync would not occur for 300 seconds, which is about 10 times as long as

(apparently) the default setting. (obviously, if the buffer space is full, a

sync would need to occur to make more room.) I've seen references of a
daemon

to periodically run a sync, or is it controlled by init?



Yes, this increases vulnerability when the data is in buffer before
committed

to disk, but in my application, this is reasonable.



_______________________________________________
freebsd-performance@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "freebsd-performance-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • Re: Abysmal disk performance, how to debug?
    ... Note that these dd "benchmarks" are completely bogus, ... will show how long it takes to actually write out the data to disk. ... your measure is wrong because time measures "dd" and sync is done ... small buffer, this should not change by more than half a second or so. ...
    (Linux-Kernel)
  • Re: giving up on 1 buffers error messsage
    ... The disk write procedures should wait for the disk to be ready, ... If, for some reason, a disk device does not empty its buffer after some ... Note that sync will not solve the problem. ... >>Since the syncer process runs periodically, can this error message be ...
    (freebsd-hackers)
  • Re: DSK to NIB Instruction
    ... > the .dsk image internally and then feed it to the emulated controller. ... > I don't have any literature available on low-level disk operation. ... This obviously depends on the emulator. ... > for a sync byte, which will be aligned correctly, then the bytes after ...
    (comp.sys.apple2)
  • Re: Discovering variable types...
    ... >memory it points to is on the heap. ... sequentially reading data, if one is randomly reading records, then a ... >project is what's prompting me to improve disk access. ... from a memory buffer I can do it in about a second. ...
    (comp.lang.pascal.delphi.misc)
  • Re: High Avg. Disk Queue Length - Memory Mapped File
    ... the buffer and give it to your device to fill. ... I shall decide the size of frame and set it to the hardware. ... there are other applications that need the ... One with Serial ATA Hard disk and another with Parallel ...
    (microsoft.public.win32.programmer.kernel)