Gestione Dat

Management Of Backups With DAT Devices Management Of Backups With DAT Devices I had the chance to use a DAT device (an old HP SuperStore DAT24) to make copy on tapes. I didn't want to install any new software to manage that task. A friend of mine gave me this guide. It was in Spanish and without any note about the author. I only translated it. So, if someone knows who is the real author, please, tell me to give him the right credits. This guide can be considered a first step to know how the thing can work. Then, everybody can customize it according to the personal needs.

1 Introduction. In this guide, I would show a simple way to make copy on tapes of your servers. The system is designed for organizations that simply need to have data accessible on a DAT tape. It involves the combination of utility mt (cpio) with rsync and some scripts. Everything described here works under GNU/Linux and, I presume, is portable to most BSD distributions. Surely, it works on my Ubuntu box.

2 Scenario. A SCSI DAT is connected to a server and the server to a network (eg Internet). Through the network and with the utility rsync, server gets the file from the servers that need to be copied to tape. Once we'll have the data in the server connected to DAT device, we'll dump those to one or more tapes. These processes are performed automatically using cron.

3 Synchronize the servers. To synchronize the servers, we will use a bash script for each server we want to synchronize. This script contains the data of the remote machine (basically the IP address) and a reference to a file which containing the files we do NOT want to synchronize. Here is an example for the serverA. In this example, we assume that serverA is accessible by name (ie, that we can resolve serverA by its IP). #!/bin/bash # rsync synchronization # Variables HOST=serverA # Use $HOST:/dir if directly from the remote host SOURCE=$HOST:/ LOCAL_DIR=/backups/serverA EXCLUDE_FILE=/usr/local/sbin/exclude-serverA

# End Variables /usr/bin/rsync --delete "$SOURCE" "$LOCAL_DIR"

-azq

--numeric-ids

--exclude-from="$EXCLUDE_FILE"

Looking at the script, we can see exactly what it does: HOST=serverA

Server name or IP to synchronize. SOURCE=$HOST:/

Directory on the remote host (serverA) to synchronize. In this example, the root. EXCLUDE_FILE=/usr/local/sbin/exclude-serverA

File that contains the files you do want to synchronize. It has to be in the following format: proc/* tmp/* dev/* mnt/*

4 Synchronization without secure access key. rsync, as used in the scripts, works through an ssh connection. The connection is encrypted and has the same characteristics as an ssh connection. One characteristic of this type of connection is the possibility of using a public key infrastructure (PKI). This system allows us to validate by private key instead of password (although the two validation systems can be combined). As the scripts are executed automatically by cron, we need a mechanism to avoid having to manually set the password and also providing security. PKI provides us that possibility. Basically, we need to create a public key on the backup server (the one connected to the DAT device) and copy it into serverB and serverA. The procedure is shown below: ssh-keygen -t dsa Generating public/private dsa key pair. Enter file in which to save the key (/root/.ssh/id_dsa): <- Enter Enter passphrase (empty for no passphrase): <- Enter Enter same passphrase again: <- Enter Your identification has been saved in /root/.ssh/id_dsa. Your public key has been saved in /root/.ssh/id_dsa.pub. The key fingerprint is: xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx root@backupsrv <- I changed this line putting the "xx"

Then we will copy the public key to each servers in the directory and with the name as indicated in the manual for ssh (1). So, we will run man 1 ssh

to be sure about the directory and name. After that, we will proceed to copy the public key to all the servers: scp /root/.ssh/id_dsa.pub serverA:/root/.ssh/authorized_keys2 scp /root/.ssh/id_dsa.pub serverB:/root/.ssh/authorized_keys2

To check we did all in the correct way, we will connect through ssh to the servers. ssh serverA

If they do NOT ask for a password, it means that we have gone well.

5 Dump the files to tape. Once synchronized the servers on the local disk of the server connected to the DAT, we have to copy the data we are interested in, to a tape. This guide aims to copy all the data on each copy. Possible other solution would be an incremental backup where, each day, we will copy only those files that have changed but I will not mention about that case here.

5.1 Periodicity The proposal is to make a dump every day. This task will be automated using cron and we would have to put the tape in the DAT device once a day. The dump data to tape should be done after the servers synchronization, which, following this model, should be done once a day prior to dump data to tape. In addition we will also copy a tape as monthly backup. In short, we will have 5 + 12 tapes in a year, one for each working day of the week and one per month. The monthly tape will be on the first Monday of each month. The process to run a cron on the first Monday of the month is not supported directly but is obtained as follows: crontab -e

Add the following line: 50 10 1-7 * *

[ "$(date +\%a)" == "Mon" ] && /usr/local/bin/monthly.sh

This will launches the first Monday of each month, the script /usr/local/bin/monthly.sh. In the example of this guide, the only thing different that we will do the first Monday of the month is to put the 'monthly' tape. However, the script monthly.sh contains a reminder message for the person responsible for putting the tape which is sent via cron. #!/bin/bash echo "Hi!" echo echo "If I have been scripted well, today is the first Monday of the month" echo "and you must to put the monthly backup tape."

5.2 Script The following script will copy the data to tape from the directory where the files are synchronized from the original server #!/bin/bash # Dump files to tape # Variables # Base dir to copy SOURCE=/backups # Logs dir LOGS=/var/log/backup # Dir to copy DIRS="serverA/var/log serverA/var/lib/mysql serverA/home/httpd" # tape device TAPE=st0 # Date format FORMAT="[%Y/%m/%d %H:%M:%S]" # End Variables # Var for logs DAY=`date --date='1 day ago' +%a` MONTHDAY=`date --date='1 day ago' +%e` # First Monday of the month? if test "$DAY" = "Mon" -a $MONTHDAY -ge 1 -a $MONTHDAY -le 7; then DAY=`date +%b` fi date=`date "+$FORMAT"` echo "----------------------" >> $LOGS/$DAY.log echo "----------------------" >> $LOGS/$DAY.err echo "$date" >> $LOGS/$DAY.err echo "----------------------" >> $LOGS/$DAY.err echo "Message from Backup Server." echo echo "Below the summary of the nightly backup." echo echo "$date Start backup" |tee -a $LOGS/$DAY.log echo "$date Rewinding the tape..." |tee -a $LOGS/$DAY.log mt -f /dev/$TAPE rewind echo "$date Starting to copy" |tee -a $LOGS/$DAY.log for dir in $DIRS; do date=`date "+$FORMAT"` echo echo "$date Started: $SOURCE/$dir" |tee $LOGS/$dir.$DAY.log tar -vzcf /dev/n$TAPE -C $SOURCE $dir >> $LOGS/$dir.$DAY.log 2>>$LOGS/$DAY.err wait $! date=`date "+$FORMAT"` echo "$date Completed: $SOURCE/$dir" |tee -a $LOGS/$dir.$DAY.log echo done echo sleep 10 echo "$date Ejecting the tape..." |tee -a $LOGS/$DAY.log mt -f /dev/$TAPE rewoffl date=`date "+$FORMAT"` echo "$date End backup" |tee -a $LOGS/$DAY.log

A little analysis to understand what the script does: SOURCE=/backups

The servers are synchronized in this directory (see section "3 Synchronize the servers") LOGS=/var/log/backup

This is the directory where the script save a log about the operation. DIRS="serverA/var/log serverA/var/lib/mysql serverA/home/httpd"

Within the directory defined in SOURCE, we choose what directories to save on tape. Each entry is separated by a space and being dumped to tape will create a brand (see Restoring files). TAPE=st0

The tape device. In /dev there are two devices st0 and nst0. The first one, automatically rewind after each action. the second, ask for an explicit rewind. We wil use both of them that's why we reference the device in that way. DAY=`date --date='1 day ago' +%a` MONTHDAY=`date --date='1 day ago' +%e`

Variables used for "extracting" the backup log's names, for example. tar -vzcf /dev/n$TAPE -C $SOURCE $dir >> $LOGS/$dir.$DAY.log 2>>$LOGS/$DAY.err

Is inside the loop which, for each directory, copy its contents to the tape in tar format and compressed (-z) in gzip format.

6 Restoring files Restoring files is as follows: 1. Enter the tape containing the files you want to restore in the DAT device. 2. go at the point (record) of the tape containing the file you need. The tape is divided into as many points (records) as directories specified in the DIRS entry into the script. In the script the variable is: DIRS="serverA/var/log serverA/var/lib/mysql serverA/home/httpd" In this case, we have 3 records: 1. serverA/var/log 2. serverA/var/lib/mysql 3. serverA/home/httpd To access a file that is under the first directory, simply rewind the tape: mt -f /dev/st0 rewind

To access a file that is under the second directory: mt -f /dev/nst0 fsf 1

Etc. 3. Now, with the command: tar xvf /dev/nst0 serverA/var/log/[filename]

for restoring a file, or tar xvf /dev/nst0 serverA/var/log

for restoring all the directory log.

7 Summaries commands to use a tape device. This small list shows only the most common options; however, it is highly recommended that you

go through man pages of mt and tar command for more options/information. To use the following comands, you have to be root. Otherwise, you can use them with sudo. Rewind tape drive: mt -f /dev/st0 rewind

Backup directory /www and /home with tar command (z - compressed): tar -czf /dev/st0 /www /home

Find out what block you are at with mt command: mt -f /dev/st0 tell

Display list of files on tape drive: tar -tzf /dev/st0

Restore /www directory: cd / mt -f /dev/st0 rewind tar -xzf /dev/st0 www

Unload the tape:

mt -f /dev/st0 offline

Display status information about the tape unit: mt -f /dev/st0 status

Erase the tape:

mt -f /dev/st0 erase

Go to previous record: mt -f /dev/nst0 bsfm 1

Forward record:

mt -f /dev/nst0 fsf 1

Go to end of data: mt -f /dev/nst0 eod

8 Conclusion As always in this case, I can NOT assure that it works for everyone. I CAN assure, it works exactly in this way with my Ubuntu box for 16 months, now, without any big problems. As I said at start, please, consider a starting guide to use a tape backup system.