Index.
09-Nov-2011: initial release. 11-Nov-2011: added amrecover warning. 12-Nov-2011: added dump/restore, removed part-cache, fixed amrecover. 02-Dec-2012: adjusted for Slackware. 19-Apr-2015: removed maintenance LAN.
Introduction.
Amanda Network
From the website: AMANDA, the Advanced Maryland Automatic Network Disk Archiver, is a backup solution that allows the IT administrator to set up a single master backup server to back up multiple hosts over network to tape drives/changers or disks or optical media. Amanda uses native utilities and formats (e.g. dump and/or GNU tar) and can back up a large number of servers and workstations running multiple versions of Linux or Unix. Amanda uses a native Windows client to back up Microsoft Windows desktops and servers.
The choice for Amanda is not so obvious, but the main reason is that Amanda uses a server/client model and writes standard tape backups with tar. It does full backups and incremental backups. Currently Amanda’s development is supported by Zmanda.
On the Home Server project Amanda will be installed on the Home Serve.r. Because client machines have a file system on the home server, the backup of the clients will be made on the home server too. Backups will be written to virtual tapes that are on an external USB disk and assembled as a tape changer with tapes. This is much more reliable and cheaper then a real tape drive and a pile of tapes. There are several backup mediums available, and you can use more then one type. Amanda can also backup remote clients if you like.
Server installation.
Build the amanda and dump packages using Slackbuild scripts and install them. Next the external USB disk needs to be formatted with a Linux compatible filesystem because they are delivered with a FAT filesystem. In this example I show a 1Tb drive:
[root@homsrv ~]# fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklab
el
Building a new DOS disklabel with disk identifier 0x04e32d0a.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
Command (m for help): n
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): p
Partition number (1-4, default 1):
Using default value 1
First sector (2048-2147483647, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-2147483647, default 2147483647):
Using default value 2147483647
Command (m for help): p
Disk /dev/sdb: 1099.5 GB, 1099511627776 bytes
255 heads, 63 sectors/track, 133674 cylinders, total 2147483648 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x04e32d0a
Device Boot Start End Blocks Id System
/dev/sdb1 2048 2147483647 1073740800 83 Linux
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[root@homsrv ~]# mke2fs -j -L WDbackup /dev/sdb1
mke2fs 1.41.14 (22-Dec-2010)
Filesystem label=WDbackup
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
67108864 inodes, 268435200 blocks
13421760 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=0
8192 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
102400000, 214990848
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 27 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
[root@homsrv ~]#
Note that I gave the disk the label WDbackup. Because it’s a USB drive it will not be certain that this disk is allways /dev/sdb1, mounting the disk with a label name does always work. Add the following lines to /etc/fstab:
# Amanda drive for virtual tapes LABEL=WDbackup /mnt/amanda ext3 defaults 1 2
Then give the following commands:
[root@homsrv ~]# mkdir /mnt/amanda [root@homsrv ~]# mount -a [root@homsrv ~]# df Filesystem Type Size Used Avail Use% Mounted on udev devtmpfs 10M 0 10M 0% /dev run tmpfs 10M 172K 9.9M 2% /run /dev/sda3 ext4 6.9G 2.0G 4.7G 30% / shm tmpfs 155M 0 155M 0% /dev/shm tmpfs tmpfs 155M 4.0K 155M 1% /tmp /dev/sda1 ext2 30M 17M 11M 61% /boot /dev/mapper/vg00-home ext3 79G 205M 75G 1% /home /dev/mapper/vg00-users ext3 79G 184M 75G 1% /mnt/users /dev/mapper/vg00-media ext3 99G 188M 94G 1% /mnt/media /dev/mapper/vg00-vserver ext3 40G 177M 38G 1% /mnt/vserver /dev/sdb1 ext3 1008G 200M 957G 1% /mnt/amanda [root@homsrv ~]#
Server configuration.
Our external drive is 1 Gb. Suppose we want to create a backup cycle of 2 weeks. This means we need about 14 tapes and some spares, so let’s try 20 tapes. According to Amanda’s website the size of each tape would be 1Gb * 0.9 / 20 = 45 Gb. I selected 40 Gb in the configuration so there is a bit more spare room available.
Another thing to think about is the number of configuration files, on a small installation like this you only need one, but more is possible. The only configuration we will create is called wpl, and just one configuration set is enough for this project. The chosen name is important because it will come back in several directories to create.
The steps to setup Amanda is to create the configuration. This file will be /etc/amanda/wpl/amanda.conf. Here you already see the first time a wpl directory appears. Then create a list of disks to backup: /etc/amanda/wpl/disklist. When both files are present, you must create the holding disks (temporary space), virtual tapes and set the correct ownership. When that is done, check it and run the first backup to see if it works as planned. Next are these steps in detail.
Configuration files.
The file /etc/amanda/wpl/amanda.conf, only the relevant parts are shown. This file is copied from /usr/share/amanda/example/amanda.conf and is well documented.
# /etc/amanda/wpl/amanda.conf at homsrv.wpl.ym
#
org "WPL UK" # your organization name for reports
mailto "admin@wpl.uk" # space separated list of operators at your site
dumpuser "amanda" # the user to run dumps under
inparallel 4 # maximum dumpers that will run in parallel (max 63)
dumporder "sssS" # specify the priority order of each dumper
taperalgo first # The algorithm used to choose which dump image to send
# to the taper.
displayunit "m" # Possible values: "k|m|g|t"
netusage 8000 Kbps # maximum net bandwidth for Amanda, in KB per sec
dumpcycle 2 weeks # the number of days in the normal dump cycle
runspercycle 14 # the number of amdump runs in dumpcycle days
# (2 weeks * 7 amdump runs per week -- all weekdays)
tapecycle 20 tapes # the number of tapes in rotation
# 2 weeks (dumpcycle) times 7 tapes per week (just
# the weekdays) plus a few to handle errors that
# need amflush and so we do not overwrite the full
# backups performed at the beginning of the previous
# cycle
bumpsize 20 Mb # minimum savings (threshold) to bump level 1 -> 2
bumppercent 20 # minimum savings (threshold) to bump level 1 -> 2
bumpdays 1 # minimum days at each level
bumpmult 4 # threshold = bumpsize * bumpmult^(level-1)
etimeout 300 # number of seconds per filesystem for estimates.
dtimeout 1800 # number of idle seconds before a dump is aborted.
ctimeout 30 # maximum number of seconds that amcheck waits
# for each client host
device_output_buffer_size 1280k
# amount of buffer space to use when writing to devices
usetimestamps yes
flush-threshold-dumped 0
flush-threshold-scheduled 0
taperflush 0
autoflush no
runtapes 2 # number of tapes to be used in a single run of amdump
# Tapechanger using virtual tapes.
tpchanger "chg-disk:/mnt/amanda/vtapes/wpl"
labelstr "WPL-.*" # tape labels
tapetype HARD-DISK # virtual tapes
maxdumpsize -1 # Maximum number of bytes the planner will schedule
amrecover_changer "changer" # amrecover will use the changer if you restore
holdingdisk hd1 {
comment "main holding disk"
directory "/mnt/users/.holding/wpl" # where the holding disk is
use -10 Gb # how much space can we use on it
# a non-positive value means:
# use all space but that value
chunksize 1Gb # size of chunk if you want big dump to be
# dumped on multiple files on holding disks
# N Kb/Mb/Gb split images in chunks of size N
}
holdingdisk hd2 {
comment "second holding disk"
directory "/home/.holding/mbse"
use -10 Gb
chunksize 1Gb
}
infofile "/var/lib/amanda/wpl/curinfo" # database DIRECTORY
logdir "/var/lib/amanda/wpl" # log directory
indexdir "/var/lib/amanda/wpl/index" # index directory
tapelist "/var/lib/amanda/wpl/tapelist" # list of used tapes
# tapetypes
define tapetype HARD-DISK {
comment "Dump onto hard disk"
length 40960 mbytes # specified in mbytes to get the exact size of 40GB
}
# dumptypes
define dumptype global {
comment "Global definitions"
}
define dumptype always-full {
global
comment "Full dump of this filesystem always"
compress none
priority high
dumpcycle 0
}
# Dumptypes for gnutar
define dumptype root-tar {
global
program "GNUTAR"
comment "root partitions dumped with tar"
compress none
index
priority low
}
define dumptype user-tar {
root-tar
comment "user partitions dumped with tar"
priority medium
}
define dumptype user-tar-span {
root-tar
comment "tape-spanning user partitions dumped with tar"
priority medium
}
define dumptype high-tar {
root-tar
comment "partitions dumped with tar"
priority high
}
define dumptype comp-root-tar {
root-tar
comment "Root partitions with compression"
compress client fast
}
define dumptype comp-user-tar {
user-tar
compress client fast
}
define dumptype comp-root {
global
comment "Root partitions with compression"
compress client fast
priority low
}
# network interfaces
define interface local {
comment "a local disk"
use 8000 kbps
}
Then /etc/amanda/wpl/disklist. Some partitions are compressed and some not. It depends on the largest part of the data that you expect is on a partition. There is also an example that excludes some directories from the backup. I usually don’t backup /var, but there may be directories on /var that you do want to backup. Use your own imagination to create this file and use the one from the example directory as a guide. The root and boot partitions are done with the dump program, the other partitions with tar.
# disklist - Amanda backups on server homsrv.wpl.ym
#
# Backups on homsrv
#
localhost root / {
comp-root
comment "homsrv root partition"
} 1 local
#
# Multimedia partition
#
localhost media /mnt/media {
user-tar
comment "homsrv Multimedia disk"
} 1 local
#
# VirtualBox machine without harddisks
#
localhost vbox /var/lib/lxc {
comp-user-tar
comment "homsrv LinuX Containers"
} 1 local
#
# All users partition
#
localhost users /mnt/users {
comp-user-tar
exclude append "./Downloads*" "./.incomplete*" "./mirror*" "./.holding*"
comment "homsrv All users directories"
} 1 local
#
# Home directories
#
localhost home /home {
comp-user-tar
exclude append "./samba/trash*" "./.holding*"
comment "homsrv Home directories"
} 1 local
To use amrecover create /etc/amanda/amanda-client.conf:
# # amanda-client.conf on homsrv # # This file normally goes in /etc/amanda/amanda-client.conf. # conf "wpl" # your config name index_server "localhost" # your amindexd server tape_server "localhost" # your amidxtaped server tapedev "" # your tape device # if not set, Use configure or ask server. # if set to empty string "", ask server # amrecover will use the changer if set to the value # of 'amrecover_changer' in the server amanda.conf. # auth - authentication scheme to use between server and client. # Valid values are "bsd", "bsdudp", "bsdtcp", "krb5", "local", # "rsh" and "ssh". # Default: [auth "bsdtcp"] auth "bsdtcp" ssh_keys "" # your ssh keys file if you use ssh auth
Create the directories and tapes.
The next shell session shows how to make all needed directories and virtual tapes. Look out that you use the directory names that belong to your configuration, and that the tape labels names match. A part must be done as root, and a part as user amanda.
[root@homsrv ~]# touch /etc/dumpdates
[root@homsrv ~]# chmod 666 /etc/dumpdates
[root@homsrv ~]# mkdir -p /mnt/users/.holding/wpl
[root@homsrv ~]# chown -R amanda:amanda /mnt/users/.holding
[root@homsrv ~]# mkdir -p /home/.holding/wpl
[root@homsrv ~]# chown -R amanda:amanda /home/.holding
[root@homsrv ~]# mkdir /var/lib/amanda/wpl
[root@homsrv ~]# chown amanda:amanda /var/lib/amanda/wpl
[root@homsrv ~]# mkdir -p /mnt/amanda/vtapes/wpl
[root@homsrv ~]# cd /mnt/amanda/vtapes/wpl/
[root@homsrv wpl]# for slot in `seq 1 20`; do mkdir slot$slot; done
[root@homsrv wpl]# cd ../..
[root@homsrv amanda]# chown -R amanda:amanda vtapes/
[root@homsrv amanda]# cd
[root@homsrv ~]# su - amanda
[amanda@homsrv ~]$ for slot in `seq 1 20`; \
do amlabel wpl WPL-$slot slot $slot; \
done
Reading label...
Found an empty tape.
Writing label 'WPL-1'...
Checking label...
Success!
Reading label...
Found an empty tape.
Writing label 'WPL-2'...
Checking label...
Success!
.....
Reading label...
Found an empty tape.
Writing label 'WPL-20'...
Checking label...
Success!
[amanda@homsrv ~]$ echo "localhost.localdomain amanda amdump" >> .amandahosts
[amanda@homsrv ~]$ echo "localhost root amindexd amidxtaped" >> .amandahosts
[amanda@homsrv ~]$ amcheck wpl
Amanda Tape Server Host Check
-----------------------------
Holding disk /mnt/users/.holding/wpl: 76352 MB disk space available, using 66112
MB
Holding disk /home/.holding/wpl: 76332 MB disk space available, using 66092 MB
slot 3: volume 'WPL-3'
Will write to volume 'WPL-3' in slot 3.
NOTE: skipping tape-writable test
NOTE: conf info dir /var/amanda/wpl/curinfo does not exist
NOTE: it will be created on the next run.
NOTE: index dir /var/amanda/wpl/index does not exist
NOTE: it will be created on the next run.
Server check took 0.732 seconds
Amanda Backup Client Hosts Check
--------------------------------
Client check: 1 host checked in 2.645 seconds. 0 problems found.
(brought to you by Amanda 3.3.0)
[amanda@homsrv ~]$ exit
logout
[root@homsrv ~]#
Testing Amanda.
Now that everything is configured and checked, create a first manual backup. As user Amanda (su – from root):
[amanda@homsrv ~]$ amdump wpl [amanda@homsrv ~]$
It can take a long time before the command completes, it depends on how much data there is to backup. When the backup is finished there should be an email that was sent to the admin user. A normal email looks like this:
Hostname: homsrv Org : WPL UK Config : wpl Date : November 9, 2011 These dumps were to tape WPL-3. The next 2 tapes Amanda expects to use are: 2 new tapes. The next 2 new tapes already labelled are: WPL-1, WPL-2 STATISTICS: Total Full Incr. Level:# -------- -------- -------- -------- Estimate Time (hrs:min) 0:01 Run Time (hrs:min) 0:02 Dump Time (hrs:min) 0:00 0:00 0:00 Output Size (meg) 21.6 21.1 0.4 Original Size (meg) 42.2 37.5 4.6 Avg Compressed Size (%) 51.1 56.3 9.2 DLEs Dumped 6 5 1 1:1 Avg Dump Rate (k/s) 860.2 1424.3 41.5 Tape Time (hrs:min) 0:00 0:00 0:00 Tape Size (meg) 21.6 21.1 0.4 Tape Used (%) 0.1 0.1 0.0 DLEs Taped 6 5 1 1:1 Parts Taped 6 5 1 1:1 Avg Tp Write Rate (k/s) 6308.6 6366.5 4340.0 USAGE BY TAPE: Label Time Size % DLEs Parts WPL-3 0:00 22M 0.1 6 6 NOTES: planner: Last full dump of localhost:root on tape WPL-3 overwritten in 1 run. planner: Last full dump of localhost:boot on tape WPL-4 overwritten in 1 run. planner: Last full dump of localhost:media on tape WPL-4 overwritten in 1 run. planner: Last full dump of localhost:vbox on tape WPL-4 overwritten in 1 run. planner: Last full dump of localhost:users on tape WPL-4 overwritten in 1 run. planner: Last full dump of localhost:home on tape WPL-4 overwritten in 1 run. planner: Full dump of localhost:home promoted from 14 days ahead. planner: Full dump of localhost:media promoted from 14 days ahead. planner: Full dump of localhost:users promoted from 14 days ahead. planner: Full dump of localhost:vbox promoted from 14 days ahead. planner: Full dump of localhost:boot promoted from 14 days ahead. taper: Slot 3 with label WPL-3 is usable taper: tape WPL-3 kb 22081 fm 6 [OK] DUMP SUMMARY: DUMPER STATS TAPER STATS HOSTNAME DISK L ORIG-MB OUT-MB COMP% MMM:SS KB/s MMM:SS KB/s -------------------------- ------------------------------------- -------------- localhost boot 0 17 17 -- 0:09 1857.1 0:03 5780.0 localhost home 0 20 4 20.2 0:05 845.8 0:00 42310.0 localhost media 0 0 0 -- 0:00 43.9 0:00 100.0 localhost root 1 5 0 9.1 0:10 41.5 0:00 4340.0 localhost users 0 0 0 81.2 0:00 169.0 0:00 650.0 localhost vbox 0 0 0 10.0 0:00 4.1 0:00 0.0 (brought to you by Amanda version 3.3.0)
Final configuration.
Now that everything works, add the following entries to the crontab of the amanda user with crontab -e -u amanda:
# crontab for amanda backup # # Check configuration 0 12 * * * /usr/sbin/amcheck -m -c -s wpl # # Perform the backup 45 1 * * * /usr/sbin/amdump wpl
Note that at 12:00 a check is done and email warnings are sent if there is something wrong. It is useful, believe me. Then every night the backup starts at 01:45.
Download.
See the download page for the script and configuration files.