Re: how to implement an append only log file?
From: Glenn Everhart (gce_at_gce.com)
Date: 11/07/04
- Next message: Rob Brooks: "Re: LNM$SYSCLUSTER_TABLE question"
- Previous message: Guy Peleg: "Re: Request for feedback - COPY utility"
- Maybe in reply to: Josef Jarousse: "how to implement an append only log file?"
- Next in thread: Josef Jarousse: "Re: how to implement an append only log file?"
- Reply: Josef Jarousse: "Re: how to implement an append only log file?"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Date: Sat, 06 Nov 2004 23:33:53 -0500
Many years ago I wrote a host program for my host-process-as-disk driver
(which has been named various things like frdriver, zrdriver...) that would
act as a write once device above a certain LBN (doing this by reading the
block, checking for a fiducial pattern on it, and allowing overwrite only
if the underlying store had the fiducial pattern). It encrypted data
written (to defend the abstraction) and used some command files I devised
and init/index=beg/head=nnn and some command files to create the index file
and directories below the fence LBN. (There were later some variants with
FDT intercepts to ensure directories never filled, since they would be
reallocated). Directories had to be preallocated bigger than default.
This beast was / is kind of a pain to set up because you have to predict
how big directories will be, but it can be used to hold log files and
encrypt them and make sure they cannot be overwritten. The FDT intercepts
also block deletion on the device and certain kinds of writing over files
that are already there, and the block write check absolutely protects any
writing in place.
This did work in tests with random log files in the sense that they got
created right, wrote to disk only once, and worked completely normally,
except of course they could not be altered. Admittedly the pointers to the
files could be trashed, but the data would still be there immutably, and
because the underlying disk encrypted the results, it was not possible to
simple clobber the container file and not make it obvious someone had been
corrupting log files. You could corrupt a file on such a thing, but it was
pretty hard to edit it so that a change could be hidden. I was well pleased
that the write-once driver level check did not prevent normal sequential
writing or appending, and that this function could be so easily had, i.e.,
by pointing the logging to the software "worm" disk. I believe this has been
on the SIG tapes, since the work was done in the early 1990s as memory serves.
I don't recall whether I ever altered the host file for alpha, but the driver
works for vax or alpha and has been available in source forever, and the
host process for vax or alpha with that driver is practically identical;
main difference being that the driver UCB definition needs to be right for
the architecture and match the driver used. One simple block cut and paste...
It's not such a bad way to do encrypted logs once you get it set
up...just not so easy to use for general storage as noted above.
In fact it is about as close to a foolproof system of its type as anything
I've ever heard of.
Glenn C. Everhart
Mike Rechtman wrote:
> Josef Jarousse wrote:
>
>> Hi,
>>
>> I have the following problem:
>>
>> I design a system and I would like it to trace each action the system
>> does. Especially it would be important that even the programm calls
>> from the admin are logged too. What I want is a log file which has an
>> owner that is different from the admin (internal audit for example)
>> and in which the application writes some informations about started
>> modules etc.. This log file should be "append-only" so that nobody can
>> delete some entries in it.
>> Is there a way to implement such a log file?
>>
>> Thanks
>> Jo
>
> Start with
> $ HELP AUDIT
>
> two gotchas:
> 1. There cannot be, by definition, anything not accessible via VMS to a
> sufficiently privileged user. Any logging you start can always be
> stopped, permanently or temporarily by the admin. (Under VMS generally
> called the system manager) - thats one of his duties.
>
> 2. Logging everything has the unfortunate result of very quickly filling
> up any disk, leading either to loss of data (earliest, latest - you
> choose) or some type of DOS.
>
> When you write "program calls": is that individual "run" commands or
> their equivalent (foreign commands, DCL commands) or do you need to go
> to the system-call level?
>
> Mike
>
- Next message: Rob Brooks: "Re: LNM$SYSCLUSTER_TABLE question"
- Previous message: Guy Peleg: "Re: Request for feedback - COPY utility"
- Maybe in reply to: Josef Jarousse: "how to implement an append only log file?"
- Next in thread: Josef Jarousse: "Re: how to implement an append only log file?"
- Reply: Josef Jarousse: "Re: how to implement an append only log file?"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Relevant Pages
|
