Re: Softupdates a mount option?

• Previous message: Marcel Moolenaar: "Re: [current tinderbox] failure on ia64/ia64"
• In reply to: Ivan Voras: "Softupdates a mount option?"
• Next in thread: Claus Assmann: "Re: Softupdates a mount option?"
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Date: Wed, 26 May 2004 23:48:22 +0200
To: Ivan Voras <ivoras@fer.hr>

On 2004-05-26 21:15 +0200, Ivan Voras <ivoras@fer.hr> wrote:
> This has been really nagging me for a long time: Why aren't softupdates
> made a mount option (like 'sync' and 'async')? Do I remember correctly that
> it is done so in NetBSD (where it's called softdeps), so it's doable?

I prepared and posted patches to make soft updates a mount option several
years ago (June 2000, in fact). I'll just append the message I received
from Kirk McKusick to the end of this Mail. In case you are interested,
I'll prepare a patch against a recent -current, tomorrow (the diff I add
to this mail is as of 4 years ago).

I'd be happy to commit my diffs (which I have kept in my system, all the
years) to -current, especially if NetBSD got that mount option 3 years ago.

Regards, STefan

>>From mckusick@mckusick.com Sat Jul 1 13:49:15 2000
Received: by hub.freebsd.org (Postfix)
        id CB4A437B9B7; Wed, 28 Jun 2000 13:51:08 -0700 (PDT)
Delivered-To: se@freebsd.org
Received: from beastie.mckusick.com (beastie.mckusick.com [209.31.233.184])
        by hub.freebsd.org (Postfix) with ESMTP
        id 8134937C1C5; Wed, 28 Jun 2000 13:51:01 -0700 (PDT)
        (envelope-from mckusick@mckusick.com)
Received: from beastie.mckusick.com (localhost [127.0.0.1])
        by beastie.mckusick.com (8.9.3/8.9.3) with ESMTP id NAA05776;
        Wed, 28 Jun 2000 13:51:01 -0700 (PDT)
        (envelope-from mckusick@beastie.mckusick.com)
Message-Id: <200006282051.NAA05776@beastie.mckusick.com>
To: arch@freebsd.org
Subject: Re: Soft updates mount change
Cc: Stefan Esser <se@freebsd.org>
Date: Wed, 28 Jun 2000 13:51:00 -0700
From: Kirk McKusick <mckusick@mckusick.com>
Content-Type: text
Status: RO
Content-Length: 12566
Lines: 301

After a long discussion (see below) with Stefan Esser, I have
agreed that it is reasonable to make soft updates a mount option
once we flip over to making them on by default. If you do not
want soft updates, you will add the option `nosoftdep' to your
fstab entry. That makes the extra options in /etc/fstab the
exception rather than the rule and generally does not require
naive administrators to know about them. It also avoids having
a flag day when all the entries in /etc/fstab need to be flipped
over as adding them now would do. I do not anticipate making
them default on until they have had wider operational experience
from the new licensing and until I resolve the problems with
using them on nearly full filesystems (writes failing because
space has not yet been freed from recently deleted files).

        Kirk McKusick

=-=-=-=-=-=-=-=

Date: Wed, 28 Jun 2000 00:38:24 +0200
From: Stefan Esser <se@freebsd.org>
To: Kirk McKusick <mckusick@mckusick.com>
Cc: Stefan Esser <se@freebsd.org>, Stefan.Esser@o-tel-o.de
Subject: Re: Soft updates mount change
Reply-To: Stefan Esser <se@freebsd.org>

On 2000-06-24 12:00 -0700, Kirk McKusick <mckusick@mckusick.com> wrote:

Please grant me a last chance to state my reasons, why I think that
soft-updates might become a mount option:

1) The pre-dominant reason to store the soft-updates status in the
   super-block is for FSCK.

2) File system update policy decisions are per file-system, not per
   medium.

3) It should not be neccessary to have single-user console access to
   change the soft-updates enable bit. (Yes, I know about tunefs on
   a partition that is mounted R/O, but that will not work in all
   situations.)

Regarding 1:

There is a requirement, that fsck is able to determine whether
the file-system was previously mounted with or with soft-updates.

This asks for a way to keep that information in a place where it
can't be deleted by accident, if the file system was not unmounted
in a clean state. Tunefs operates on unmounted file-systems, whether
clean or not. That means, that the SOFTDEP but in fs_flags can be
modified either way on an unclean file system, as long as it is
not mounted.

The decision to use soft-updates should not share a bit with the
status flag that controls fsck's operation.

Regarding 2:

My policy to use or not use soft-updates depends on the file-system,
not the partition or medium (consider removable media). How about a
ZIP drive that is used with soft-updates in my system, but is written
to and disconnected without a clean unmount on a system, that does not
have soft-updates in the kernel (e.g. FreeBSD-2.2.8). The medium will
require a fsck on my system, and fsck must assume it was last written
to with soft-update dependencies enforced, when in reality it was not.

This is a constructed case (I do not even own a ZIP drive), but meant
to explain my point of view, that the use of soft-updates does not
depend on the medium but rather on the file system.

My policy currently states, that soft-updates be used on all file
systems except "/" (since I had bad luck with /bin being only partly
populated after a "make installworld", which had required three times
the free space required by the new binaries).

I might have a policy, that requires /var to be written "sync", which
kind of conflicts with soft-updates.

I may have removable media that are moved between a number of systems
and mounted on different mount points, with changing access and usage
policies incl. whether async, sync or soft-updates are suitable for
some particular use.

All these points aks for soft-updates to be configured per mount-point,
not per disk partition (i.e. *not* in the super-block).

Regarding 3:

I have a lot of systems in remote locations, some with only limited
access to the console. (For example, there may be a modem that is
switched off, unless activated for the duration of a remote access.)

If I want to enable soft-updates on tens of machines, I'd rather only
distribute a configuration file, than connect to each machine, boot
into single-user mode, enable soft-updates on each file-system with
tunefs, and then let the system go back to multi-user.

There are other scenarios, where a system may receive a completely new
kernel, without an administrator present, who knows how to use tunefs.
(Think about PicoBSD: It should be possible to have soft-updates enabled
or disabled by a configuration file delivered with the boot floppy.)

The natural place for persistent per file-system configuration is either
the super-block, where disklabel or newfs store immutable parameters,
that can't be modified without loosing all data on a partition.

The other place where such information is kept is /etc/fstab, which
holds the usage, security and update policy decisions of a system in
per records per mount-point. (Even if the first column in fstab is the
partition name, I consider fstab to specify what to mount on a *mount
point* and how to do it. but I admit that this is just my personal POV ;-)

It is of course possible the have /etc/rc (or one of the rc.* sub-tasks)
modify soft-updates before the "mount -a". But that would be more complex
than my proposed solution.

> Your above proposal would work, though that is not how NetBSD
> implemented it. I feel that it is a lot of extra mechanism for very
> little gain. Administrators generally make a one-time decision to
> run soft updates on a filesystem. It is not the sort of thing that
> they want to change on a regular basis. It is possible to run tunefs

Agreed. But there are multiple places where this information may be
stored. One is to hide it in the super-block, the other to expose it
in /etc/fstab. Since mount reports the soft-updates state, I think the
POLA asks for mount to be able to choose between softdep/nosoftdep.

> on a filesystem that is mounted read-only, so it no more difficult
> to use tunefs than it is to make it a mount-time option (i.e., they
> still have to down-grade to read-only, set the option, then upgrade).

This is not easily possible for some file-systems (like /var), where
some processes keep files open for writing at all times and may fail,
if the file system mode is forced to R/O.

> Finally, I expect that soft updates will eventually just be defaulted
> to `on' when a filesystem is built, and in a few rare instances an
> administrator will want to turn it off. I do not want to have an
> option that needs to be added to nearly every fstab entry to get
> the default behavior.

I fully agree that the default should not have to be added to each
line in fstab. But then it could still default to on in the kernel,
and a "nosoftdep" could disable it for selected file systems. But I
see your point.

Hmmm, I still do not think that the command character of FS_DOSOFTDEP
and the reporting character (was previously mounted with soft-updates)
of the same bit are a good design choice. The two functions should be
separated. One way to do this is by making soft-updates a mount option
as I propose, the other to have two flag bits in the super-block, one
enable soft-updates, the other to report to fsck whether it was enabled.

> Plus it is just one more bit of trivia that
> new system administrators need to learn to make their systems run
> well. The more of those details that need not be learned because
> they just do the right thing, the better.

But the tunefs method is anything but obvious. Well, if the next version
of "sysinstall" offers to enable soft-updates in the partition editor,
it will not be missed as easily as currently ...

I think that soft-updates should be on by default, except possibly on the
root file-system (which may happen to be too small: at least 15MB of free
space are required for a "make installworld" or part of the programs in
/bin and /sbin will be lost). There should of course not be different
defaults for / and other file-systems, since this would violate the POLA.
I'm still not convinced. I think that soft-updates should be enabled by
default, and some people may want to disable it on the root fs ...

I am writing this message on a system, which is modified to make softdep a
mount option. This required a total of some 10 additional lines (incl.
comments) to be added to ffs_vfsops.c, plus a few lines that changed from
fs_flags to mnt_flag. A total of 3 lines had to be added to the mount program
(two to enable/disable softdep, one to print its state for "mount -p").

Patches are appended below, just to show that this is a very minor change.
Please consider adding these patches at least at the time, when soft-updates
defaults to "on" ...

Best regards, STefan

*****************************************************************************
Patches to make softdep a "mount" option:

-> Initial mount (no update mount, file system currently not mounted):

The following patches make ffs_mount() perform a soft-updates mount
(which sets the FS_DOSOFTDEP bit in fs->fs_flags, if successful), if
all of the following conditions are true:

1) the MNT_SOFTDEP flag is set in mp->mnt_flag
2) soft-updates is compiled into the kernel
3) the file system is unmounted or mounted read-only

The FS_DOSOFTDEP bit is cleared, if the mount succeeds without
MNT_SOFTDEP being set. This can be caused by either of:

1) a successful mount without the "-o softdep" option (or the
   nosoftdep option, if softdep is to become the default).
2) a mount on a system without soft-updates support

-> Update mount (typically: read-only -> read-write):

If this is an update mount and the file-system is not clean, then the
soft-updates enable status is "locked" (which is an even stronger
condition than the requirement, that the file system be unmounted
for "tunefs -n enable", which doesn't prevent changing an un-clean
file system and mis-directing a subsequent fsck run ...).

(If the file system is mounted R/O, the situation is identical to
the "Initial mount" case above.)

--- /sys/ufs/ffs/ffs_softdep.c 2000/06/22 00:29:53 1.69
+++ /sys/ufs/ffs/ffs_softdep.c 2000/06/27 18:22:49
@@ -962,5 +962,5 @@
 
         mp->mnt_flag &= ~MNT_ASYNC;
- mp->mnt_flag |= MNT_SOFTDEP;
+ fs->fs_flags |= FS_DOSOFTDEP;
         /*
          * When doing soft updates, the counters in the
--- /sys/ufs/ffs/ffs_softdep_stub.c 2000/06/16 13:00:33 1.11
+++ /sys/ufs/ffs/ffs_softdep_stub.c 2000/06/27 18:15:44
@@ -72,4 +72,5 @@
         struct ucred *cred;
 {
+ mp->mnt_flag &= ~MNT_SOFTDEP;
 
         return (0);
--- /sys/ufs/ffs/ffs_vfsops.c 2000/06/04 04:50:36 1.122
+++ /sys/ufs/ffs/ffs_vfsops.c 2000/06/27 20:33:37
@@ -196,4 +196,12 @@
                 err = 0;
                 ronly = fs->fs_ronly; /* MNT_RELOAD might change this */
+ if (ronly == 0) {
+ /* preserve current softdep status, unless mounted read-only */
+ if (fs->fs_flags & FS_DOSOFTDEP) {
+ mp->mnt_flag |= MNT_SOFTDEP;
+ } else {
+ mp->mnt_flag &= ~MNT_SOFTDEP;
+ }
+ }
                 if (ronly == 0 && (mp->mnt_flag & MNT_RDONLY)) {
                         flags = WRITECLOSE;
@@ -244,5 +252,5 @@
 
                         /* check to see if we need to start softdep */
- if (fs->fs_flags & FS_DOSOFTDEP) {
+ if (mp->mnt_flag & MNT_SOFTDEP) {
                                 err = softdep_mount(devvp, mp, fs, p->p_ucred);
                                 if (err)
@@ -658,4 +666,7 @@
                 goto out;
         }
+ /* clear softdep flag in superblock, if not a softdep mount */
+ if (mp->mnt_flag & MNT_SOFTDEP == 0)
+ fs->fs_flags &= ~FS_DOSOFTDEP;
         ump = malloc(sizeof *ump, M_UFSMNT, M_WAITOK);
         bzero((caddr_t)ump, sizeof *ump);
@@ -746,5 +757,5 @@
                 fs->fs_maxfilesize = maxfilesize; /* XXX */
         if (ronly == 0) {
- if ((fs->fs_flags & FS_DOSOFTDEP) &&
+ if ((mp->mnt_flag & MNT_SOFTDEP) &&
                     (error = softdep_mount(devvp, mp, fs, cred)) != 0) {
                         free(base, M_UFSMNT);

--- /usr/src/sbin/mount/mntopts.h 1999/10/09 11:54:07 1.16
+++ /usr/src/sbin/mount/mntopts.h 2000/06/27 19:06:19
@@ -57,4 +57,5 @@
 #define MOPT_NOCLUSTERW { "clusterw", 1, MNT_NOCLUSTERW, 0 }
 #define MOPT_SUIDDIR { "suiddir", 0, MNT_SUIDDIR, 0 }
+#define MOPT_SOFTDEP { "softdep", 0, MNT_SOFTDEP, 0 }
 
 /* Control flags. */
--- /usr/src/sbin/mount/mount.c 2000/01/15 14:28:13 1.39
+++ /usr/src/sbin/mount/mount.c 2000/06/27 20:14:39
@@ -741,4 +741,5 @@
         if (flags & MNT_NOSYMFOLLOW) res = catopt(res, "nosymfollow");
         if (flags & MNT_SUIDDIR) res = catopt(res, "suiddir");
+ if (flags & MNT_SOFTDEP) res = catopt(res, "softdep");
 
         return res;
--- /usr/src/sbin/mount/mount_ufs.c 1999/08/28 00:13:27 1.16
+++ /usr/src/sbin/mount/mount_ufs.c 2000/06/27 17:26:50
@@ -67,4 +67,5 @@
         MOPT_SYNC,
         MOPT_UPDATE,
+ MOPT_SOFTDEP,
         { NULL }
 };

_______________________________________________
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"

• Previous message: Marcel Moolenaar: "Re: [current tinderbox] failure on ia64/ia64"
• In reply to: Ivan Voras: "Softupdates a mount option?"
• Next in thread: Claus Assmann: "Re: Softupdates a mount option?"
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]