How to backup MySQL databases, web server files to a FTP server automatically with Shell Script
Part I :System + Web + MySQL backup script
This is a simple backup solution for people who run their own web server and MySQL server on a dedicated box or VPS. Most dedicated hosting provider provides the backup service using NAS or FTP servers. These service providers will hook you to their redundant centralized storage array over private VLAN. Since I manage couple of boxes, here is my own automated solution.
Making incremental backups with tar
You can make tape backups. However, sometime tape is not an option. GNU tar allows you to make incremental backups with -g option. For example following command will make incremental backup of /Data/apps,/Data/webapps, /home, and /etc directories:
Where,
- -g: Create/list/extract new GNU-format incremental backup and store information to /var/log/tar-incremental.log file.
Making MySQL databases backup
mysqldump is a client program for dumping or backing up mysql databases, tables and data. For example following command displays the list of databases:
Output:
Enter password:
cwiki tags ajax mysql phpads snews live csns myspaces workspaces
Now you can backup each database with mysqldump command:
A simple backup system plan
The main advantage of using FTP or NAS backup is a protection from data loss. You can use various protocols to backup data:
- FTP
- SSH
- RSYNC
- Other Commercial solutions
However, I am going to write about FTP backup solution here. The idea is as follows:
- Make full backup every Sunday night i.e. backup everything every Sunday
- Next backup only those files that has been modified since the full backup (incremental backup)
This is a seven-day backup cycle.
Our sample setup
Local Server ===>>> Remote Server[ftp/nas server] IP:211.10.10.10 ===>>> 221.10.10.10
Let us assume that your ftp details are as follows:
- FTP server IP: 221.10.10.10
- FTP Username: username
- FTP Password: passwd
- FTP Directory: /211.10.10.10/bak (or /)
You will store data at remote server as follows:
=>> /211.10.10.10/bak/full/mm-dd-yy/files – Full backup
=>> /211.10.10.10/bak/incremental/mm-dd-yy/files – Incremental backup
Automating tasks of backup with tar
Now you know how to backup files and mysql databases using tar and mysqldump commands respectively. It is time to write a shell script that will automate entire procedure.
- First script will collect all data from both MySQL database server and from file system to temporary directory called /backup using tar command
- Next, script will login to ftp server and create a directory structure at remote server as discussed above
- Script will dump all files from /backup to ftp server
- Script will remove temporary backup from /backup
- Script will send you an email notification if ftp backups failed due to any reason.
You must have following command installed:
- ncftp ftp client
Note: Build and Install ncftp client
1.wget ftp://ftp.ncftp.com/ncftp/ncftp-3.2.0-src.tar.gz
2.$ tar zxvf ncftp-3.2.0-src.tar.gz
3.$cd ncftp-3.2.0
4.$make&&make install
- mysqldump command
- GNU tar command
Here is the sample script:
# System + Web + MySQL backup script
# Full backup day - Sun (rest of the day do incremental backup)
# Copyright (c) 2005-2006 nixCraft
# This script is licensed under GNU GPL version 2.0 or above
# Automatically generated by http://bash.cyberciti.biz/backup/wizard-ftp-script.php
# ———————————————————————
### System Setup ###
DIRS="/Data/apps /Data/webapps /home /etc"
BACKUP=/tmp/backup.$$
NOW=$(date +"%d-%m-%Y")
INCFILE="/root/tar-inc-backup.dat"
DAY=$(date +"%a")
FULLBACKUP="Sun"</font>
MUSER="root"
MPASS="passwd"
MHOST="localhost"
MYSQL="$(which mysql)"
MYSQLDUMP="$(which mysqldump)"
GZIP="$(which gzip)"</font>
FTPD="/211.10.10.10/bak//incremental"
FTPU="username"
FTPP="passwd"
FTPS="221.10.10.10"
NCFTP="$(which ncftpput)"</font>
EMAILID="<a href="mailto:longrujun@gmail.com">longrujun@gmail.com</a>"</font>
[ ! -d $BACKUP ] && mkdir -p $BACKUP || :</font>
if [ "$DAY" == "$FULLBACKUP" ]; then
FTPD="/211.10.10.10/bak//full"
FILE="fs-full-$NOW.tar.gz"
tar -zcvf $BACKUP/$FILE $DIRS
else
i=$(date +"%Hh%Mm%Ss")
FILE="fs-i-$NOW-$i.tar.gz"
tar -g $INCFILE -zcvf $BACKUP/$FILE $DIRS
fi</font>
# Get all databases name
DBS="$($MYSQL -u $MUSER -h $MHOST -p$MPASS -Bse 'show databases')"
for db in $DBS
do
FILE=$BACKUP/mysql-$db.$NOW-$(date +"%T").gz
$MYSQLDUMP -u $MUSER -h $MHOST -p$MPASS $db | $GZIP -9 > $FILE
done</font>
#Start FTP backup using ncftp
ncftp -u"$FTPU" -p"$FTPP" $FTPS<<EOF
mkdir $FTPD
mkdir $FTPD/$NOW
cd $FTPD/$NOW
lcd $BACKUP
mput *
quit
EOF</font>
if [ "$?" == "0" ]; then
rm -f $BACKUP/*
else
T=/tmp/backup.fail
echo "Date: $(date)">$T
echo "Hostname: $(hostname)" >>$T
echo "Backup failed" >>$T
mail -s "BACKUP FAILED" "$EMAILID" <$T
rm -f $T
fi</font>
Setup a cron job
Just add a cron job as per your requirements:
Part II :System+Web files backup script
It is time to write a shell script that will automate entire procedure.
- First script will collect all data from both web and from file system to temporary directory called /backup using tar command
- Next, script will login to ftp server and create a directory structure at remote server as discussed above
- Script will dump all files from /backup to ftp server
- Script will remove temporary backup from /backup
- Script will send you an email notification if ftp backups failed due to any reason.
Here is the sample script:
#!/bin/sh
# System + MySQL backup script
# Full backup day – Sun (rest of the day do incremental backup)
# Copyright (c) 2005-2006 nixCraft
# This script is licensed under GNU GPL version 2.0 or above
# Automatically generated by http://bash.cyberciti.biz/backup/wizard-ftp-script.php
# ———————————————————————
### System Setup ###
DIRS=”/Data/apps /Data/webapps /home /etc”
BACKUP=/tmp/backup.$$
NOW=$(date +”%d-%m-%Y”)
INCFILE=”/root/tar-inc-backup.dat”
DAY=$(date +”%a”)
FULLBACKUP=”Sun”
###Create Archives###
FTPD="/211.10.10.10/bak//incremental"
FTPU="username"
FTPP="passwd"
FTPS="221.10.10.10"
NCFTP="$(which ncftpput)"</font>
EMAILID="<a href="mailto:longrujun@gmail.com">longrujun@gmail.com</a>"</font>
[ ! -d $BACKUP ] && mkdir -p $BACKUP || :</font>
if [ "$DAY" == "$FULLBACKUP" ]; then
FTPD="/211.10.10.10/bak//full"
FILE="fs-full-$NOW.tar.gz"
tar -zcvf $BACKUP/$FILE $DIRS
else
i=$(date +"%Hh%Mm%Ss")
FILE="fs-i-$NOW-$i.tar.gz"
tar -g $INCFILE -zcvf $BACKUP/$FILE $DIRS
fi</font>
# Get all databases name
DBS="$($MYSQL -u $MUSER -h $MHOST -p$MPASS -Bse 'show databases')"
for db in $DBS
do
FILE=$BACKUP/mysql-$db.$NOW-$(date +"%T").gz
$MYSQLDUMP -u $MUSER -h $MHOST -p$MPASS $db | $GZIP -9 > $FILE
done</font>
#Start FTP backup using ncftp
ncftp -u"$FTPU" -p"$FTPP" $FTPS<<EOF
mkdir $FTPD
mkdir $FTPD/$NOW
cd $FTPD/$NOW
lcd $BACKUP
mput *
quit
EOF</font>
if [ "$?" == "0" ]; then
rm -f $BACKUP/*
else
T=/tmp/backup.fail
echo "Date: $(date)">$T
echo "Hostname: $(hostname)" >>$T
echo "Backup failed" >>$T
mail -s "BACKUP FAILED" "$EMAILID" <$T
rm -f $T
fi</font>
<h3>Setup a cron job</h3>
Just add a cron job as per your requirements:
<code><font face="Courier New">5 0 * * * /Data/admin/shell/ftpbackup_1.sh >/dev/null 2>&1</font></code>
<code></code>
</font>
没有评论▼