Re: "secure" file flag?

From: Stefan Eßer (se_at_FreeBSD.org)
Date: 11/22/03

  • Next message: Duane H. Hesser: "Re: integer and long max/min values" 
    Date: Sat, 22 Nov 2003 00:56:07 +0100
To: Wes Peters <wes@softweyr.com>

    On 2003-11-21 14:09 -0800, Wes Peters <wes@softweyr.com> wrote:
    > As for performance, you really need to flush the on-device cache on each
    > pass to make sure the bit patterns get written to the platter in proper
    > order. I don't see any clever way to coalesce the writing of the
    > various patterns to multiple blocks short of a kernel thread, either,
    > so performance would be abysmal. Imagine removing a large file,
    > overwriting each block in 37 (IIRC) passes, syncing all the way through
    > the on-disk cache after *every block.*

    I may be way off, but I do not think, that a special thread or
    a cache flush after each block is required:

    A simple algorithm could just mark each buffer with a special
    kind of dirty flag and a counter for the pass number (in fact,
    the existing dirty flag could be used, and a counter set to the
    number of passes required, with 0 indicating that the buffer is
    to be flushed to disk "as is" in the normal way).

    All blocks belonging to the file must be read from disk, if not
    already present in some buffer, for the erasure process to start.
    (See below for large files.)

    In preparation of the first pass, the data in the buffer could
    be complemented and then written back to disk as in a normal dirty
    buffer flush.

    After some block has been written, its counter will be updated
    and the block will still be marked dirty, until the number of
    passes required to consider the data irrecoverable has been
    reached. (Blocks that had the counter set to 0 at the start of
    the flush are normal writes and need not be considered, here.)

    On successive passes, the block can be overwritten with specific
    bit patterns or random data according to some procedures for secure
    erasure of data from magnetic media.

    On each turn of the elavator algorithm, a cache flush is initiated,
    if the previous path included at least one buffer with the "secure
    erase" flag set. (Any number of unrelated reads or writes could have
    been executed during this path of the elevator.)

    After the specified number of iterations, the individual block will
    be marked as free and can be put into the free list. Until then, it
    can not be accessed or reclaimed.

    That way, the normal operation of the disk would not be impacted.
    On each pass of the elevator algorithm, each buffer flagged to be
    securely erased will be written once and its state will be updated.

    No special cleaner thread is required, just a byte for the counter.

    The biggest problem could be the first step of the algorithm (to
    read all file blocks into buffers that are marked dirty and get the
    number of the first erasure pass assigned).

    But I guess, that the erasure of long files could be broken in
    several parts, that each fit into the RAM available for buffers.
    Only this case would require that the unlink or process exit be
    delayed until the last part of the file has been read in and
    marked dirty and to be overwritten.

    Regards, STefan
    _______________________________________________
    freebsd-hackers@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
    To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"

  • Next message: Duane H. Hesser: "Re: integer and long max/min values"

    Relevant Pages

    • Re: Response.Flush: Differences between IIS 6.0 and 5.0?
      ... since IIS 6.0 is now on top of http.sys. ... smaller buffer and so on, each flush cause the packet to send.... ... > Server: Microsoft-IIS/5.0 ...
      (microsoft.public.inetserver.iis)
    • Re: Can anyone explain why this is happening?
      ... Is it possible that your writes do not include a local buffer flush ... written file (to guarantee that newly written data have been flushed to ... before it reads from a new file (which also flushes all buffered written ...
      (comp.parallel.mpi)
    • Re: Problems with sys.stout.flush()
      ... Carl Banks wrote: ... console until it sees a newline unless you flush the buffer. ... option for python and a few examples from this sitehttp://stackoverflow.com/questions/107705/python-output-buffering ...
      (comp.lang.python)
    • Re: File output buffer overrun?
      ... > old IBM mainframe Fortran 77 compiler to a modern Linux system. ... a 4K buffer, so it sounds like you're getting far enough into the code ... to flush the buffer ONCE but then the code crashes before the next ... more calcs ... ...
      (comp.lang.fortran)
    • Re: 2.6.18 ext3 panic.
      ... There is finds dirty buffer attached to the comitting transaction. ... set_page_dirtyand all buffers in the page are marked dirty. ... The first process now wakes up, locks the buffer, clears the dirty bit ... adding to array pid 1836 ...
      (Linux-Kernel)