(no subject)



I will soon be installing an Areca ARC-1110 and 3x 1.5TB Seagate
Barracuda SATAs into a 3.2GHz Northwood P4 with 1GB of RAM, and I'm
wondering which would be the most stable filesystem to use.

I've read the bigdisk page [1] and the various information about ZFS on
the FreeBSD Wiki [2]. I'm aware of the tuning requirements that ZFS
requires, and upgrading to 4GB of RAM would be quite possible as it was
understood beforehand that ZFS requires a large quantity of it.

My questions are as follows.

1. I'm aware of the fact that ZFS works better on 64-bit platforms, and
that alone has me thinking that it's not a good fit for this particular
machine. But apart from that, it seems that ZFS is not yet stable
enough for my environment (only about 25 users but in production
nonetheless). To me, [3] paints all sorts of ugly pictures, which can
be summarized as "count on ZFS-related panics and deadlocks happening
fairly regularly" and "disabling ZIL in the interest of stability will
put your data at risk." Comments about live systems using ZFS (on
7.0-RELEASE or 7-STABLE) would be appreciated.

I'm using 7.0-RELEASE/amd64 with ZFS on several machines without any stability problems. Here are the configs (prefetch is disabled for performance reasons):

- with 1GB RAM (probably with just 1GB RAM system would be faster using UFS2 instead of ZFS)
vfs.zfs.prefetch_disable=1
vm.kmem_size="512M"
vfs.zfs.arc_max="150M" (at first it was 200M, but lots of swapping made me reduce it)

- with 2GB RAM:
vfs.zfs.prefetch_disable=1
vm.kmem_size="950M" (could be higher, even 1536M, but then there is not much RAM left for your apps)

- with 8GB RAM:
vfs.zfs.prefetch_disable=1
vm.kmem_size="1536M" (this could probably be higher: up to 2047M, but I haven't tried it).

General rule to make it stable is to make the difference between vm.kmem_size and vfs.zfs.arc_max larger. vfs.zfs.arc_max is by default 3/4 of vm.kmem_size.
You can achieve it by making vm.kmem_size bigger (but this leaves less memory for your applications) or reducing vfs.zfs.arc_max (but this reduces performance, since less memory will be available for caching).
Problem with stability comes when kmem usage is at it's peak. arc_max is just a value after some of it will be deleted. But in some cases (high I/O activity) it will grow faster than the thread that reduces it (to a size less than arc_max) can delete.

2. [1] appears to be a bit dated. Nevertheless, I'm inclined to think
that the status described there (as well as in various man pages) still
applies to UFS2 on 7.0-RELEASE. Please correct me if I'm wrong or let
me know if the state of affairs has improved significantly in 7-STABLE.
2a. Does the information contained in [1] apply to ZFS as well?

[1] is outdated.
GEOM, GPT, UFS2 and ZFS are safe to use for many hundreds of terabytes.
What is limited is MBR partitioning used by fdisk (2TB limit).

3. As the array will be for data only and not be booted, will it be
possible to use fdisk to slice it up, or will I need to use gpt?

fdisk can be used for slicing for disks that are up to 2TB. But I would recomment to use GPT (which doesn't have this limit) instead. There is no reason not to.

4. My planned course of action will be to attempt to newfs the device
itself (da0, all 3TB of it) or 1 full-disk slice (da0s1). Failing that,
I will attempt to gconcat da0s1 and da0s2 (1.5TB each), although I
suspect that may not work since for one thing, growfs is not yet 64-bit
clean. In either case, I'm very interested in using gbde/geli to
encrypt the fs. If either of these paths are not possible or
recommended, are there any suggestions for alternate means of creating a
3TB fs?

If you will go for the ZFS route instead of UFS2 then don't make one logical array from your disks (da0) in your RAID controller, but instead make one logical array to be one phisical disk (so you have da0, da1 and da2) so you can use ZFS RAID functionality instead.

If you are going to use UFS2 then you must use gjournal for disks of that size (or you will have background fscks that can last for ages and die a horrible death saying not enough memory).




_______________________________________________
freebsd-questions@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscribe@xxxxxxxxxxx"



Relevant Pages

  • Re: recommended memory for zfs
    ... I heard zfs is very ram hungry. ... I'm looking to buy more ram now. ... I believe I heard a calculation of 1GB cache per 1TB of disk. ... Was just see if I should just get raid controller and more ram down the road. ...
    (freebsd-stable)
  • Re: recommended memory for zfs
    ... I heard zfs is very ram hungry. ... I'm looking to buy more ram now. ... I believe I heard a calculation of 1GB cache per 1TB of disk. ... Was just see if I should just get raid controller and more ram down the road. ...
    (freebsd-stable)
  • Re: Is ZFS production ready?
    ... data is quietly corrupted by RAM. ... and checksumming doesn't get any errors, and do write wrong data with right ... that you are unlikely to get big data corruption. ... ZFS survived 2 crashes. ...
    (freebsd-questions)
  • Re: zpool scrub tank && high file system activity caused crash
    ... I am running FreeBSD 7.0-RELEASE with GENERIC on a AMD XP Athlon 3500+ with ... I changed the size the kernel was allowed to be to at least 512 M as that is recommended, and I got the same panic as the first email I sent out, I also ran out of space on /var so it did not save the vmcore:(. ... I am going to wait till I get my 64 bit system up and running before messing with ZFS again. ... I don't think the amount of ram I have in that machine would be enough with some of the other stuff it will have to do as well, if I let the kernel eat up to a GB. ...
    (freebsd-hackers)
  • UFS2 or ZFS for 3TB disk?
    ... wondering which would be the most stable filesystem to use. ... I'm aware of the tuning requirements that ZFS ... possible to use fdisk to slice it up, or will I need to use gpt? ... fact work, would doing so effectively work around the userland tool ...
    (freebsd-questions)