Re: My project wish-list for the next 12 months

From: Andre Oppermann (andre_at_freebsd.org)
Date: 12/02/04

  • Next message: Mike Silbersack: "Re: HD Mirroring"
    Date: Thu, 02 Dec 2004 23:37:06 +0100
    To: Scott Long <scottl@freebsd.org>
    
    

    Scott Long wrote:
    > Stephan Uphoff wrote:
    >
    >> On Thu, 2004-12-02 at 09:41, Andre Oppermann wrote:
    >>
    >>> The holy grail of course is to mount
    >>> the same filesystem 'rw' on more than one box, preferrably more than
    >>> two.
    >>> This requires some more involved synchronization and locking on top
    >>> of the
    >>> cache invalidation. And make sure that the multi-'rw' cluster stays
    >>> alive
    >>> if one of the participants freezes and doesn't respond anymore.
    >>>
    >>> Scrolling through the UFS/FFS code I think the first one is 2-3 days of
    >>> work. The second 2-4 weeks and the third 2-3 month to get it right.
    >>> If someone would throw up the money...
    >
    > Although I don't know the specifics of your experience, I can easily
    > imagine how hard it would be to make this work on UFS. Common
    > operations like walking a file path to the root are nearly impossible to
    > do reliably without an overbearing amount of synchronization. Then you
    > have all of the problems synchronizing buffered data and metadata.

    You don't do it the fully synchronized way. The semantics will be somewhat
    like with NFS. Reading in the filesystem does not invoke synchronization.
    Only writing, or actually intending to write something, requires synchro-
    nization with all other machines in the filesystem cluster. Synchronization
    ensures that writes to a directory or file are properly serialized and that
    caches are invalidated at the same time.

    > Softupdates would be a nightmare, if not impossible.

    To the contrary. It provides a nice place to link code into and it helps
    to keep performance up to good levels. Any inode changes entering the
    softdep code would cause a message to all other machines informing them
    of the change. If another one wants to update the same inode or something
    dependend on it it has to wait until you have written it to disk. His
    backnotification would of course move this inode to the front of the softdep
    work queue to not stall your request.

    As long as you don't want to have mmap() work across machines with contents
    in memory but not yet written to disk things stay pretty much sane.

    -- 
    Andre
    _______________________________________________
    freebsd-hackers@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
    To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
    

  • Next message: Mike Silbersack: "Re: HD Mirroring"

    Relevant Pages

    • Re: My project wish-list for the next 12 months
      ... Reading in the filesystem does not invoke synchronization. ... nization with all other machines in the filesystem cluster. ... If another one wants to update the same inode or something ... in memory but not yet written to disk things stay pretty much sane. ...
      (freebsd-current)
    • Re: Not enough parallelism in programming
      ... All synchronization in current machines is memory based. ... exponent of memory traffic interference, not much can be done at the ...
      (comp.arch)
    • Re: ATA problems again ... general problem of ICH7 or ATA?
      ... After some days of testing, disk disappear, system reboots itself, ... resynchronize gmirror and work for next few days till the next disk lose. ... whole synchronization is done after more then 5 hours (the longest was ...
      (freebsd-stable)