Re: Filesystem snapshots dog slow

On Tue, Oct 16, 2007 at 06:54:11AM -0500, Eric Anderson wrote:
The problem is the way the snapshots work in UFS2.

[...]

... the docs that say 'incredibly fast' are actually referring to
small filesystems, that are not busy (with writes). Maybe the docs
should be clarified for now. You can submit patches to the docs you
found that say that if you'd like to help out.

I'll work on that this week. I'm not sure if I read it in the FreeBSD
Handbook or the FAQ or someplace on the web. If it's in the lesser two,
I'll send-pr changes.

I recommend reading up a little bit on how the snapshots for UFS2
work. It will give you a good understanding of what the issue is.
Essentially, your disk is hammered making copies of all the cylinder
groups, skipping those that are 'busy', and coming back to them later.
On a 200Gb disk, you could have 1000 cylinder groups, each having to
be locked, copied, unlocked, and then checked again for any subsequent
changes. The stalls you see are when there are lock contentions, or
disk IO issues. On a single disk (like your setup above), your
snapshots will take forever since there is very little random IO
performance available to you.

I understand the majority of what you've written, and this definitely
gives me a better insight to what the problem is. I hope that others
will eventually find this thread and find their answer here. Thank you
for providing this.

That said, for such setups, would you recommend *not* using snapshots?
If so, possibly we should consider removing the following code from
src/sbin/dump/main.c:

334 } else if (snapdump == 0) {
335 msg("WARNING: %s\n",
336 "should use -L when dumping live read-write "
337 "filesystems!");
338 } else {

FreeBSD 7 includes ZFS. Have you thought about using it?

I haven't. For starters, I keep seeing mails from users reporting data
corruption or kernel panics when using it. This doesn't mean ZFS is
"bad" (it's likely ZFS is exposing data corruption for them that's
occuring at a lower level (controller, disk, or RAM)), but it does keep
me from considering it a worthy alternative to UFS2 on production
systems at this time. And then there's this:

WARNING: ZFS is considered to be an experimental feature in FreeBSD.

This doesn't give admins "warm fuzzies". :-) I'm considering trying ZFS
on my home system (a ZFS filesystem atop a gstripe(8) pair), where I
also perform nightly backups using dump, but there's a part of me which
is asking "just how important is your data? What if ZFS breaks in the
middle of you using dump(8) on it? What then?"

The problem isn't that developers don't know the problem exists, or
that they don't have hardware, or a serial console access to a system.
The problem is that there are only so many developers, and so much
time, and this is a big mountain to climb. It's hard to find an
experienced person to do the work (for free), when they could be doing
anything else they wish. I think, that in the end, for some of these
aging issues to get resolved, there needs to be another bounty put out
on it. I think rsync.net might even have one started for this issue
already - you might think about adding to the bounty, or officially
offering hardware through there.

I'll take a peek at the site. I have no problem donating some money
and/or hardware to the cause, but as I'm one person, I don't have an
infinite supply of cash to donate to projects like this. About the
most I could donate would be US$1000 (and that's a bit extreme), which
wouldn't provide for very much by itself.

--
| Jeremy Chadwick jdc at parodius.com |
| Parodius Networking http://www.parodius.com/ |
| UNIX Systems Administrator Mountain View, CA, USA |
| Making life hard for others since 1977. PGP: 4BD6C0CB |

_______________________________________________
freebsd-hackers@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@xxxxxxxxxxx"

Relevant Pages