Re: Need some small help on shell script - delete old files

On Jun 15, 3:46 pm, "steven_nospam at Yahoo! Canada"
<steven_nos...@xxxxxxxx> wrote:
On Jun 14, 11:39 am, David <david4...@xxxxxxxxxxx> wrote:





On Jun 4, 5:02 pm, "steven_nospam at Yahoo! Canada"

<steven_nos...@xxxxxxxx> wrote:
On Jun 4, 11:01 am, David <david4...@xxxxxxxxxxx> wrote:

1.) I do not have subdirectories past the /home/tom64/public_html/cgi-
bin/arp3/backup   directory.   Please revise and confirm the best
command so that it does not search subdirectories past this directory
if that is important.

2.) Please look at my original script from the first message to this
post.   For some reason, my script has not been gzipping these .sql
files.  Any idea how to fix that so that they do get gzipped?

Thank you so very much!-

Here is what I would do if using your setup. Keep in mind, I tend to
use piped commands because my workload would rarely suffer from
performance issues, but some people may have a better way.

# START OF SAMPLE #
###################
BKPPATH=/home/tom64/public_html/cgi-bin/arp3/backup/
BKPFILE=$(/bin/date +"%Y-%b-%d").sql
#
# Do the current mysqldump
/usr/bin/mysqldump --opt -utom64 -pPaSSwoRD tom64 > ${BKPPATH}$
{BKPFILE}
#
# gzip any files that start with a number and end with sql
# this is sufficiently unique in the BKPPATH
cd ${BKPPATH}
ls | grep "^[0-9].*sql$" | xargs gzip
#
# Remove oldest backups from specified path - do NOT go into
subdirectories
# Anything older than 7 days with .sql.gz extension will be removed
# For testing purposes - you may replace "rm" command with "ls"
find ${BKPPATH} -name "*.sql.gz" -mtime +7 -print |
   grep -v "${BKPPATH}.*/" |
   xargs /bin/rm
#################
# END OF SAMPLE #

The find command shows all files in the directory and subdirectories.
We then use grep -v to limit our view to files in the designated
directory by saying ignore any files that have the BKPPATH name along
with one or more characters (.*) and another slash (/). The results of
the grep are then sent to xargs to issue the rm command. The extra
gzip of $FILENAME at the end is not necessary as far as I can tell
because the ls will see that new file and gzip it in the ls | xargs
section.

CAVEAT: This was tested briefly on an IBM AIX 5.3 system in Korn Shell
and worked for me. It should naturally be tested on a test system
before being released to any production server. You may also want to
add additional checks, such as making sure your BKPPATH is set,
directory is not empty, etc. as I have not tried all possible
scenarios. Use at your own risk.

Regards,

SteveN

Dear SteveN,

I am running the script and have noticed that the file names have been
a bit strange.  Getting (at times) double dumps at different times
during the same day.  Could this possibly be from this script?   I
have this script running on cron at 3:45am:
45 3 * * * pathtoscript

here is a list of the files in the backup directory:

-rw-r--r--    1 growthtr growthtr 16668440 Jun 11 14:15 2009-
Jun-11-14-14-59.sql.gz
-rw-r--r--    1 growthtr growthtr 16609994 Jun 11 03:45 2009-
Jun-11.sql.gz
-rw-r--r--    1 growthtr growthtr 16694367 Jun 12 14:18 2009-
Jun-12-14-18-19.sql.gz
-rw-r--r--    1 growthtr growthtr 16662433 Jun 12 03:45 2009-
Jun-12.sql.gz
-rw-r--r--    1 growthtr growthtr 124439331 Jun 13 15:08 2009-
Jun-13-15-08-20.sql
-rw-r--r--    1 growthtr growthtr 16716441 Jun 13 03:45 2009-
Jun-13.sql.gz
-rw-r--r--    1 growthtr growthtr 124621863 Jun 14 03:45 2009-
Jun-14.sql

Any idea why I am getting two dumps like this annd that the dump files
with the longer file name are at irregular times?

Also, any idea why 2009-Jun-14.sql  did not gzip from last night?

Thanks you!- Hide quoted text -

- Show quoted text -

I have no explanation. If you use the date format of $(date +"%Y-%b-
%d"), then the date will emerge as YYYY-Mmm-DD. In your case, it looks
like it is running at a different time (between 2-3pm) and including
hour/minute/second info. Could it be that you have another script
still running (either in cron or as an AT job) that runs the backup in
the afternoon?

Also not sure why the qzip did not run on last night's backup. Could
be a timing issue (file was still locked by the backup routine when
gzip tried to access it?). The good thing with the routine is that
even if it misses that June 14 back, on June 15, the gzip should
compress it because of the ls and xargs combo.

Sorry I could not be of more help.- Hide quoted text -

- Show quoted text -


By the way, that brings up ANOTHER issue...What if you wanted to run
more than one backup per day? You would probably need to include the
hour and minutes in the backup name so you could see which one was
which and not clobber the previous backup you did that same day. In
which case, you want to change the date command to $(date +"%Y-%b-%d-%H
%M").

Keep in mind you can use any date format you prefer, but I would avoid
any slashes, backslashes, colons, and spaces. Those tend to be
reserved or difficult to deal with in filenames. I generally use
YYYYMMDD-HHMMSS if I want a unique name for my backups, and the
numbers make an easier sort order.
.

Relevant Pages

  • Re: file backup in windows
    ... I want my script to identigy the user ... logged in and go to the resp outlook folder or should be able to read ... operating system to try a few sample commands at the command line. ...
    (comp.lang.python)
  • Re: compress command
    ... Let's say you want to use the gzip command on the RedHat system. ... Match this path against the path in your script, and make sure they are the same. ...
    (RedHat)
  • Re: Need some small help on shell script - delete old files
    ... command so that it does not search subdirectories past this directory ... The find command shows all files in the directory and subdirectories. ... gzip of $FILENAME at the end is not necessary as far as I can tell ... I am running the script and have noticed that the file names have been ...
    (comp.unix.shell)
  • Re: backup script directory errors fixed, now it cant find the eof
    ... I run the script and it finds a premature end of file on line 53? ... The:command not founde 21,30,31 are just blank lines. ... # full and incremental backup script ... # The rest of the time an incremental backup is made. ...
    (comp.unix.shell)
  • Re: remote server backup script
    ... > Just type away at the command line. ... Make a backup of whatever you want ... > command line, in a script. ... >>the server ssh key on the clients machine). ...
    (comp.os.linux.misc)