Re: Do too many files hurt a directory?

From: Lew Pitcher (Lew.Pitcher_at_td.com)
Date: 03/26/04 

Date: Fri, 26 Mar 2004 12:58:42 -0500

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Charlie Gibbs wrote:

> A process went wild at a customer site the other night, and started
> creating files with gay abandon. When the staff came in the next
> morning, the system was extremely sluggish. I dialed in and found
> that over 280,000 files had been created, all in the same directory.
> All of the files belong to root (the user kicked off the process's
> parent under the wrong ID), and most of them are empty or nearly so.
> I finally managed to delete all of the files (it took the better part
> of a day), but the system is still sluggish - even a simple ls command
> takes 10 seconds or more to respond.

Depending on the filesystem in question, filename location /may/ be
implemented as a linear search of the directory. In this case, I'd
expect that the system would be very sluggish whenever the overloaded
directory is read.

However, I thought that 'modern' filesystems implemented various
directory search improvements that eliminated the linear search.

> Is a directory somehow stretched out of shape if it has an absurdly
> high number of files created? If so, what's the best way to recover?

Probably by moving the remaining files out of the current directory and
into a new directory. Once that's done, remove the old directory
completely, and move the new directory into it's place. This should
improve things by re-optimizing the directory search.

> Currently I have the customer creating a new directory and copying
> the contents of the old directory into it. Then she can delete the
> old directory and rename the new one.

See above. We seem to think alike <grin>

> We've got our fingers crossed -
> the way the system is running it's going to take a long time - but if
> it fails, what then? Can fsck clean things up, or is it time for a
> full reformat a la Windows?

Gak! You Always Have Other Options. We'll think of something.

- --
Lew Pitcher, IT Consultant, Enterprise Application Architecture
Enterprise Technology Solutions, TD Bank Financial Group

(Opinions expressed here are my own, not my employer's)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (MingW32)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFAZG9RagVFX4UWr64RAtrmAJ9yesqGGSqtuzeDnlILBlvkz6B84QCgqlH+
Cr216c2714BkfA1AKXSUQj8=
=rQ6G
-----END PGP SIGNATURE-----