Re: UFS2 limits

On Sun, Nov 09, 2008 at 01:40:51AM +0000, no-spam@xxxxxxxxxxxxx wrote:
Hi,
I have a FreeBSD server that has about 10,500 subdirectories within a single
directory.
This number will keep rising and I assume UFS2 has a limit to the number of
sub-directories in a single directory - can anyone tell me what it is?

As far as I know, there is no such limit on the number of files/dirs
inside of a directory.

I don't want to change the topic of discussion, but I *highly* recommend
you ***stop*** whatever it is you're doing that is creating such a
directory structure. Software which has to iterate through that
directory using opendir() and readdir() will get slower and slower as
time goes on.

If this is something you've written or have control over or can work
with engineers in regards to, I recommend you change your directory
naming scheme to have separate subdirectories with the first 2 or 3
letters of the directory you wish to create. E.g.:

/some/place/00/00ilikezeros/*
/some/place/01/01binaryheaven/*
/some/place/aa/aardvarks/*
/some/place/ab/abuse/*
/some/place/ac/actuary/*
...
/some/place/xy/xylophones/*

You get the point.

Traversing this structure is much more efficient, and requires very
little code change on your part. Those who run nameservers that host
many zones, for example, use this structure to ensure the daemon doesn't
take 32498231 years to start up.

What about ZFS?

At some point I'll have to re-arrange things so that I have a deeper directory
structure, just wondering when I'll hit the limit so I can plan in advance :-)

What baffles me is why you're looking at this problem from a " "how can
the filesystem solve this engineering mistake I made for me" standpoint,
rather than "how can I solve this engineering mistake I made so that it
doesn't impact the filesystem". Very strange. Sometimes looking at
things in a different light makes all the difference.

Hope this helps.

P.S. -- I hope this mail makes it to you, because your From line is
no-spam@xxxxxxxxxxxxx (I'll be surprised if your account name really is
that!).

--
| Jeremy Chadwick jdc at parodius.com |
| Parodius Networking http://www.parodius.com/ |
| UNIX Systems Administrator Mountain View, CA, USA |
| Making life hard for others since 1977. PGP: 4BD6C0CB |

_______________________________________________
freebsd-questions@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • NFS : -alldirs requested but is not a filesystem mountpoint
    ... The server has got a filesystem located at /data ... This local filesystem has several subdirectories, ... subdirectories to be visible to a client host, but not all as a whole. ... proprietary or legally privileged information. ...
    (freebsd-questions)
  • How to efficiently remove large directory tree
    ... A oracle process has tried to dump a core but created recursively a very ... The amount of subdirectories is estimated to be almost 100,000 looking ... at the increase of used inodes on the filesystem. ... Problem is that most unix utilities fail on these huge path lenghts; ...
    (comp.unix.solaris)
  • How to efficiently remove large directory tree
    ... A oracle process has tried to dump a core but created recursively a very ... The amount of subdirectories is estimated to be almost 100,000 looking ... at the increase of used inodes on the filesystem. ... Problem is that most unix utilities fail on these huge path lenghts; ...
    (comp.sys.hp.hpux)
  • How to efficiently remove large directory tree
    ... A oracle process has tried to dump a core but created recursively a very ... The amount of subdirectories is estimated to be almost 100,000 looking ... at the increase of used inodes on the filesystem. ... Problem is that most unix utilities fail on these huge path lenghts; ...
    (comp.unix.shell)
  • Re: OT: MoinMoin and Mediawiki?
    ... > isn't workable even with subdirectories. ... then the filesystem does that pretty well, ... For backlink indexing, that's a relatively easy index to maintain ... you'll want already-existing code to do it for ...
    (comp.lang.python)