Re: No of files in a directory (performance)

From: Fredrik (paltskallen_at_hotmail.com)
Date: 03/03/04 

Date: 3 Mar 2004 07:26:28 -0800

OK.

I have noticed that rm -rf on a dir takes way longer on a dir with
many files in it, ex if there are 10k files it takes 10 sec, but with
70k files it takes 150 sec.

find|xargs rm
takes the same time (per file that is). It seems that -rf does some
magic that slows it down.

Bob Harris <harris@zk3.dec.com> wrote in message news:<harris-8A4C82.20384627022004@cacnews.cac.cpqcorp.net>...
> In article <ff81d3b0.0402270129.205a3e@posting.google.com>,
> paltskallen@hotmail.com (Fredrik) wrote:
>
> > Hello!
> >
> > We have an application that creates and reads a lot of files.
> > These files are splitted on 100 dirs. Currently there are ~70 000 files in
> > each dir.
> > Are there any limit that should be considered regarding to performance
> > issues??
> >
> > Using advfs and san disks.
>
> AdvFS in Tru64 UNIX v5.0 or later uses b-tree indexing for directory
> access. While a huge directory does take slightly longer to create,
> lookup, delete a file, it is _NOT_ linear. The performance is really
> very good.
>
> However, if you ask 'ls' to display all the files it will take a long
> time to fetch all the files, then sort them, and then display them, but
> if you use 'ls -f' it skips the sorting, which can make things faster.
>
> Bob Harris

Relevant Pages

  • Re: real time sort results fetched from different sites
    ... Currently, it uses perl to fetch the data, sort it and display it. ... so that if you can qualify the requests enough, then after the first user makes that request, the next user gets a cached copy and the new parse starts AFTER they have already received the ...
    (comp.lang.javascript)
  • Re: Change report sort order
    ... order you want to sort and set the sorting ... then make them not display later when it's working properly. ... Sorting numbers and sorting text is decidedly ... > Tom Ellison ...
    (microsoft.public.access.queries)
  • Re: D3/Linux program abort
    ... same sort of fire could occur anywhere. ... Most developers are ... error message so that we can get through diagnostics quickly and get ... There's also the "techno error" that developers tend to display when ...
    (comp.databases.pick)
  • RE: Query sorting help
    ... result is to sort by press column first then sort found data by date. ... pieces for Press1 and press3. ... I wish to display the data first by Field "Press1" if data exists display it ...
    (microsoft.public.access.queries)
  • Re: oldest American skull found!
    ... > Trigger was a horse, you dummy, not his great aunt. ... It's sort of like the bone reliquiae of various saints on ... > display in churches and museums all over Italy. ... > important enough to have his/hers bones displayed in a museum. ...
    (sci.archaeology)