Re: too many files
From: Aaron Morris (aaronmorris_at_MINDSPRING.COM)
Date: 07/30/03
- Previous message: John Jolet: "Re: make copy of savevg tape"
- In reply to: Nguyen, Joseph: "too many files"
- Next in thread: Hunter, Mark: "Re: too many files"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Date: Tue, 29 Jul 2003 21:34:04 -0400 To: aix-l@Princeton.EDU
If your network link is sufficiently fast, try not compressing
everything (or you could tell compress not to compress data as
aggressively by using -1). Depending on the link speed and the CPU
power, the majority of your time might be spent on compression . Also,
using the find command can be a bottleneck since the backup command must
rely on the find command for a list of files. Using something like tar
or pax directly could possibly speed up the process.
ie.
tar cf - /indirectory | rsh remotehost "tar xf -"
or if you still need compression:
tar cf - /indirectory | compress -c | rsh remotehost "uncompress -c |
tar xf -"
Also, if you have a reliable link, take a look at rsync
http://rsync.samba.org
Nguyen, Joseph wrote:
> I have a filesystem that contains 900,000+ files in one directory. I ran
> the following command to copy files to another host and it ran for couple
> days and stops. Event just run the find command would take a long time.
>
> find /indirectory -print | backup -iqf- | compress -c | rsh remotehost
> "(uncompress -c | restore -xf- )"
>
> Do you know any other command that can speed up the copy? we try to backup
> to tape and restore and that also take days.
>
> Joseph
>
-- Aaron W Morris <aaronmorris@mindspring.com> (decep)
- Previous message: John Jolet: "Re: make copy of savevg tape"
- In reply to: Nguyen, Joseph: "too many files"
- Next in thread: Hunter, Mark: "Re: too many files"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Relevant Pages
- Re: Nmon scheduling question
... Runs "gzip" to compress it. ... # Script to generate the spreadsheet,
compress, & email daily logfiles ... # generated by the daily nmon command run by the cron.
... (AIX-L) - Recoevering an over-written / lost presentation
... I over-wrote my presentation when trying to compress it ... from the command
line; I accidentally inverted the flags of the 'tar' ... (microsoft.public.mac.office.powerpoint) - Over-written / lost presentation
... I over-wrote my presentation when trying to compress it ... from the command
line; I accidentally inverted the flags of the 'tar' ... (microsoft.public.powerpoint) - Re: De-compress a file from UNIX
... > compressed using the UNIX compress command. ... However, if you
know in advance that you have access to WinZip on the host, ... the you could easily
command WinZip to decompress the file. ... (microsoft.public.dotnet.languages.csharp) - Re: find... tar.. gzip...
... > I'm trying to use find command to compress a lot of files. ... >
And delete source files. ... Search for 'tar takes last file from a find', ...
(comp.unix.shell)