Re: "tar" command has error messages.

Robert.Frank_at_unibas.ch
Date: 10/10/03 

Date: 10 Oct 2003 09:00:49 +0100

Not just really, really close, I'd say. Considering the amount of bytes
beeing backed up (2339697Kbytes or 2395849728bytes > 2^31!)then it is
clear that he has the problem of file size limitation. The fact that it
stops 2048 bytes short of the maximum allowed is due to the tar command
and the buffering (it uses buffers >= 2Kbytes) so the last buffer cannot
be written before the 2G limit is crossed.
With 2.5.1 Logan will have to find another solution.

Robert

Logan Shaw <lshaw-usenet@austin.rr.com> wrote:
> Pom wrote:
> > And size of tar file that can"t finish backup.
> >
> > # ls -l /backup/home.tar
> > -rw------- 1 root other 2147481600 Oct 10 10:56 /backup/home.tar
>
> That's really, really close to the largest file you can have that
> uses 32-bit offsets:
>
> $ echo '2^31 - 2147481600' | bc -l
> 2048
> $
>
> I would say it's highly likely you're running into that issue.
>
> Unfortunately, you are running 2.5.1, which is something like 6 or 7
> years old. I can't recall whether it has support for 64-bit files
> (large files). Here are some things you could try:
>
> (1) "man mount_ufs" and see if describes a "largefiles" mount option.
> If so, remount with that option. You can add it to /etc/vfstab,
> or do "mount" to see the current options, then do
> "mount -o remount,largefiles,(otheroptions) /backup" to
> remount it with large files.
>
> (2) In case tar isn't largefile aware but the shell is, you could
> try this command instead:
>
> tar cf - export/home > /backup/home.tar
>
> If this works, it will be because it has the shell open the
> file (and pass the special flags to open() to allow large
> files) and then allows tar to write to standard output as
> if it's a stream, in which case size doesn't matter.
>
> Of course, #2 requires that you have mounted your filesystem
> so that it can handle large files.
>
> If neither of those works, you could of course upgrade. Or,
> another way of dealing with backups would be to write the
> backup to several smaller files. Here's a little shell script
> I call "breakup" that will let you do that:
>
> #! /bin/sh
>
> prefix="$1"
> piece=0
> filename="$prefix"."$piece"
> filelist="$prefix".list
> > "$filelist"
>
> while :
> do
> echo 1>&2 "writing to '$filename'"
> dd obs=128k count=8192 of="$filename"
> if [ `ls -l "$filename" | awk '{print $5}'` -eq 0 ]
> then
> rm "$filename"
> break
> fi
> echo "$filename" >> "$filelist"
>
> piece=`expr $piece + 1`
> filename="$prefix"."$piece"
> done
>
> All you'd have to do is something like this:
>
> cd / && tar cf - export/home | breakup /backup/home.tar
>
> This should create /backup/home.tar.0 up through /backup/home.tar.1,
> each with as size of 1 GB or less. You can then use "cat" to put
> them back together again, or at least I think you can. (Presumably
> "cat" can handle several separate 1 GB files when writing to stdout.)
> Anyway, the command should look like this:
>
> cat `cat /backup/home.tar.list` | tar tvf -
>
> Naturally, this is not something I've tried before. The best solution
> would be to upgrade, and I make no guarantee that my code won't
> trash your backups. (In particular, there's one way that it could --
> if any of those commands reads from stdin when you're not expecting
> it, it will absorb some of the input that should be going to an
> output file!) But anyway, maybe it will make it possible to do
> backups until you get the chance to upgrade, which would be valuable...
>
> - Logan
> 

-- 
Departement Informatik             tel  +41 (0)61 267 14 66
Universitaet Basel                 fax. +41 (0)61 267 14 61
Robert Frank                                        
Klingelbergstrasse 50   Robert.Frank@unibas.ch (NeXT,MIME mail ok)  
CH-4056 Basel           (remove any no_spam_ from my return address)
Switzerland             http://www.informatik.unibas.ch/personen/frank_r.html

Relevant Pages

  • Re: "tar" command has error messages.
    ... stops 2048 bytes short of the maximum allowed is due to the tar command ... > In case tar isn't largefile aware but the shell is, ... > If neither of those works, you could of course upgrade. ... > backups until you get the chance to upgrade, ...
    (comp.unix.admin)
  • Re: "tar" command has error messages.
    ... stops 2048 bytes short of the maximum allowed is due to the tar command ... > In case tar isn't largefile aware but the shell is, ... > If neither of those works, you could of course upgrade. ... > backups until you get the chance to upgrade, ...
    (comp.unix.solaris)
  • Re: "tar" command has error messages.
    ... > And size of tar file that can"t finish backup. ... you could of course upgrade. ... another way of dealing with backups would be to write the ... the command should look like this: ...
    (comp.unix.admin)
  • Re: "tar" command has error messages.
    ... > And size of tar file that can"t finish backup. ... you could of course upgrade. ... another way of dealing with backups would be to write the ... the command should look like this: ...
    (comp.unix.solaris)
  • Re: "tar" command has error messages.
    ... > And size of tar file that can"t finish backup. ... you could of course upgrade. ... another way of dealing with backups would be to write the ... the command should look like this: ...
    (comp.sys.sun.admin)