Re: [OT] Tales from the server room

From: Benjamin Gawert (bgawert_at_gmx.de)
Date: 11/23/04 

Date: Tue, 23 Nov 2004 06:54:34 +0100

Massimiliano Adamo wrote:

> My apologies for my English, not really nice, but this is the real
> story of myself today and how I got absolutely frustrated.
>
> - A ticket about an implementation on an HP-UX server came to our
> team, and I saw it so easy to do that I decided to run the task in a
> flash. There were few patch to install and the patch were already
> residing on a depot server.
> - What can be easier to do! The ticket was just telling: installing
> these patches prior reboot. This normally means that a reboot is
> required to get the change operational. I went thru SAM, looked for
> the depot server, went to directories, I said ok, yes, I accept
> everything and proceeded with all the installation steps.
> - Just a strange popup took my attention for a while: "after
> installation a reboot is required". For me it still means: reboot is
> required to get the change operational …

No, this is not Windows! It simply means "You made a bunch of changes, and
the system has to be rebooted in order to work reliable and as expected"...

> - The installation just finish and I get an horrible popup with just
> one option: "OK", and one label saying: "click OK to reboot".
> - I got terrified: I realized in that moment that this server was one
> of the most important machines, serving the whole Germany operations!
> - But I thought: I have one more chance, I leave this windows open
> until reboot time and I put a stick on my client!
> - I was about happy, but I didn’t know what pain in the …. was coming
> - … a colleague of mine comes ….
> - I show to him what bull*** I have done …
> - Colleague tenderly says: you have to kill SAM and everything will be
> ok. Don't worry for nothing!
> - ME: are you sure I can do it without worries?
> - Colleague: suuuuuurrrre
> - ME: Ok …. I'm going to kill SAM.
> - kill -KILL xxxx, bla bla bla. I immediately get a popup message:
> system is going down for reboot now !!!
> - in less then one minute the big boss appeared behind the glasses of
> our offices and I saw the end closest to me. I immediately told about
> my mistake and so on. He noticed 2 mistakes in one (don't know why)
> and calmly said: on third mistake you’re just out.
> - I think it was not 100% of my fault. The implementation task didn’t
> tell about this "installation feature" (and honestly I didn’t
> imagine), my colleague never thought to shut his mouth just in time
> and told me to kill sam and I was paying all the price alone.

As much as I can understand Your frustration, I really don't see the problem
here...

First, sam (in this case the menu-driven swinstall) clearly shows a warning
message after the analyzing phase saying that this patch requires a kernel
rebuild and thus the machine needs to be rebootet, and it asks You if You
want to continue with the installation process if a patch requires a reboot.
So even before the real installation starts You should know that a reboot is
required, and if this is not possible, You just can answer "no" on the
question if the installation should be started...

Second, You said that the server was very important for Your company. So I
don't understand why two people without enough knowledge are messing around
during business time with it. Patches are applied at times no-one needs the
machine, and not when they are busy. During working hours no-one touches
critical equipment except if really really necessary.

IMHO, You're very lucky. If You had done this in our companys server room,
You certainly would have lost Your job...

Benjain