I was kinda against using S3 for backups or anything as it has a cost. By rotating my backups keeping under the gb store limit I manage to only pay for transfers and that's always under 1$ USD. Now I have 4 weeks of backups stored on S3 thanks to the below script that will :

  • Put the file to S3
  • rotate the 4th oldest backup file and delete it
  • email me the results.

First set up an S3 account, second setup s3cmd tool (http://s3tools.org/s3cmd)

Edit the following to suit :

LLOC=/var/log
PFILE=`/bin/cat /tmp/curzmbk`
BKBKT="s3://basstech/backups/"
DATE=`date +%Y-%m-%d`
BKLOC="/backup"
MAILADDRESS@SERVER</pre>

So here is the script to do it :-

#!/bin/bash

LLOC=/var/log/backups
#PFILE=$1
PFILE=`/bin/cat /tmp/curzmbk`
BKBKT="s3://basstech/backups/"
DATE=`date +%Y-%m-%d`
BKLOC="/backup"


function loghd() {
  echo "Copy and file rotation to S3" > $LLOC/S3-CPY-$DATE.log
  echo "Started at:- `date +%c`" >> $LLOC/S3-CPY-$DATE.log
  echo "--------------------------------------------" >> $LLOC/S3-CPY-$DATE.log
}

function lstbkt() {
  /usr/bin/s3cmd ls $@
}

function putflbkt() {
  echo  "Putting File $@" >> $LLOC/S3-CPY-$DATE.log
  /usr/bin/s3cmd put $BKLOC/$PFILE $@ >> $LLOC/S3-CPY-$DATE.log
  echo "Status  $?" >> $LLOC/S3-CPY-$DATE.log
}

function rotatebk() {
  echo "--------------------------------------------" >> $LLOC/S3-CPY-$DATE.log
  /usr/bin/s3cmd ls $BKBKT > /tmp/bkbktlst
  echo "Current Files in $BKBKT" >> $LLOC/S3-CPY-$DATE.log
  cat /tmp/bkbktlst >> $LLOC/S3-CPY-$DATE.log
  BKCNT=`/usr/bin/wc -l /tmp/bkbktlst | awk ' { print $1 }'`
  if [ $BKCNT -ge 3 ]; then
    echo "--------------------------------------------" >> $LLOC/S3-CPY-$DATE.log
    OLDBK=`cat /tmp/bkbktlst | sort | sed -n 1p | awk '{print $4}'`
    echo "Roating out oldest backup file:-" >> $LLOC/S3-CPY-$DATE.log
    echo "$OLDBK" >> $LLOC/S3-CPY-$DATE.log
    /usr/bin/s3cmd del $OLDBK  >> $LLOC/S3-CPY-$DATE.log
    echo "Status :- $?" >> $LLOC/S3-CPY-$DATE.log
    rm -f /tmp/bkbktlst
  else
    echo "Less then two backup files, not roating" >> $LLOC/S3-CPY-$DATE.log
    rm -f /tmp/bkbktlst
  fi
}

function lognd() {
  echo "--------------------------------------------" >> $LLOC/S3-CPY-$DATE.log
  echo "Copy to S3 ended on `date +%c`" >> $LLOC/S3-CPY-$DATE.log
  echo "--------------------------------------------" >> $LLOC/S3-CPY-$DATE.log
}

function mailrpt() {
  /bin/cat $LLOC/S3-CPY-$DATE.log | /bin/mail -s "S3 Copy Report for `date +%c`" admin@basstech.net
}


loghd
putflbkt  $BKBKT
rotatebk
lognd
mailrpt