Re: No of files in a directory (performance)

From: Fredrik (paltskallen_at_hotmail.com)
Date: 03/03/04 

Date: 3 Mar 2004 07:26:28 -0800

OK.

I have noticed that rm -rf on a dir takes way longer on a dir with
many files in it, ex if there are 10k files it takes 10 sec, but with
70k files it takes 150 sec.

find|xargs rm
takes the same time (per file that is). It seems that -rf does some
magic that slows it down.

Bob Harris <harris@zk3.dec.com> wrote in message news:<harris-8A4C82.20384627022004@cacnews.cac.cpqcorp.net>...
> In article <ff81d3b0.0402270129.205a3e@posting.google.com>,
> paltskallen@hotmail.com (Fredrik) wrote:
>
> > Hello!
> >
> > We have an application that creates and reads a lot of files.
> > These files are splitted on 100 dirs. Currently there are ~70 000 files in
> > each dir.
> > Are there any limit that should be considered regarding to performance
> > issues??
> >
> > Using advfs and san disks.
>
> AdvFS in Tru64 UNIX v5.0 or later uses b-tree indexing for directory
> access. While a huge directory does take slightly longer to create,
> lookup, delete a file, it is _NOT_ linear. The performance is really
> very good.
>
> However, if you ask 'ls' to display all the files it will take a long
> time to fetch all the files, then sort them, and then display them, but
> if you use 'ls -f' it skips the sorting, which can make things faster.
>
> Bob Harris

Relevant Pages

  • Re: real time sort results fetched from different sites
    ... Currently, it uses perl to fetch the data, sort it and display it. ... so that if you can qualify the requests enough, then after the first user makes that request, the next user gets a cached copy and the new parse starts AFTER they have already received the ...
    (comp.lang.javascript)
  • Re: Sort order - Data midified / Created
    ... Under "Date formats" there is a drop down menu that you click called "Short date:" it is here you can choose the format you would like to have your dates display in. ... you can change or add many different heading in columns to view and sort the different properties that different apps will let you set when you fill in the fields in the properties sheet for the file you have created and or add your own Custom properties if allowed depending on the application. ... That being said there are many ways you can display the information about files and folders from within Windows Explorer from Stacking, filtering, grouping and using the search box at the top of Windows Explorer. ... Open control panel type "Folder options" in the dialog that opens under the search tab check the box next to use natural language search. ...
    (microsoft.public.windows.vista.general)
  • RE: Address books
    ... "LOAMA" wrote: ... trying to get the alphabetical sort? ... "Duane Hookom" wrote: ... Then when I went to properties to display new page there was only properties ...
    (microsoft.public.access.reports)
  • Re: Change report sort order
    ... order you want to sort and set the sorting ... then make them not display later when it's working properly. ... Sorting numbers and sorting text is decidedly ... > Tom Ellison ...
    (microsoft.public.access.queries)
  • Re: D3/Linux program abort
    ... same sort of fire could occur anywhere. ... Most developers are ... error message so that we can get through diagnostics quickly and get ... There's also the "techno error" that developers tend to display when ...
    (comp.databases.pick)