Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
HOWTO: Automated server-client backup with rsync
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
BoZ
Tux's lil' helper
Tux's lil' helper


Joined: 27 Aug 2003
Posts: 114
Location: The Netherlands

PostPosted: Tue Jun 01, 2004 10:39 pm    Post subject: HOWTO: Automated server-client backup with rsync Reply with quote

It's the first time I'm making a little howto, but I'll try to make it readable :)


Setup:
There's a server and a desktop, both running gentoo. It can be any distribution actually, but since we're on the gentoo-forums...! In this howto I'll show you how to make a backup from the servers etc-directory to the desktop and the other way around.
Since the server is a server, it runs the rsync-daemon (which can be combined with a local portage-mirror).


General configuration

First create /backup /backup/server and /backup/desktop on both the server and the desktop:
Code:
mkdir /backup /backup/server /backup/desktop
In /backup/server the server will store all data that needs to be backed-up, then it will be rsynced to the desktop. Same story for the desktops data in /backup/desktop.


Server configuration:

First we're setting up the rsync-daemon, the configuration-file is located in /etc/rsync/rsyncd.conf
Our configuration will look like this:
Code:
use chroot = no
max connections = 1
hosts allow = 192.168.0.2
pid file = /var/run/rsyncd.pid
motd file = /etc/rsync/rsyncd.motd
transfer logging = yes
log format = %t %a %m %f %b
syslog facility = local3
timeout = 300

[server-backup]
path = /backup/server
comment = The server backup
lists = false

[desktop-backup]
path = /backup/desktop
comment = The desktop backup
read only = false
uid = root
gid = root
list = false

The important parts:
Max connections is set to 1 since there's no need for two syncs.
Hosts allow is set to the desktop's ip ONLY. There's no need to add more ip's if you're not using rsyncd for anything else!

In both backup-modules, listing is turned off, this is because we will only have a script using these rsync-entries, hiding it might prevent any snooping.

The desktop-backup module also needs some explaining.
Read only is disabled simply because the desktop will have to write down the backup.
The uid and gid are both set to root because by default rsync will store the files as uid and gid nobody. Since we're backing up important data, we don't want anybody else then root to be able to touch any of it.

Of course rsyncd should be running, start it with:
Code:
/etc/init.d/rsyncd start

And add it to the default runlevel:
Code:
rc-update add rsyncd default


That's it for rsyncd


Now we'll have to put that data that needs a backup in /backup/server
In this example I'll only backup /etc but you can add anything you like.

This method makes a daily tar of /etc
Edit your crontab:
Code:
crontab -e

Then add this line:
Code:
0 7 * * * /usr/local/sbin/backup 1>&2

Add this in /usr/local/sbin/backup
Code:
#!/bin/bash
tar -czpf /backup/server/etc-server-`date +%d-%m-%Y`.tar.gz /etc

Don't forget to make it executable:
Code:
chmod 700 /usr/local/sbin/backup

This script will tar and gzip your etc-directory every day at 07:00 into a file named etc-server-dd-mm-yy.tar.gz in /backup/server
Of course you can change the cronjob, a daily backup for /etc might be over the top for example.
Making a daily tar has the advantages that we have several backups, not just the latest.

If you settle for the latest backup or the files to backup are to large to tar and store several times (daily!) you can just skip the line for your crontab and symlink /etc
Code:
ln -s /backup/server/etc /etc


Of course you only need one of these 2 solutions.

That's it for the server, you now have a rsync-daemon ready for syncing and the first backup ready to be transferred


Desktop configuration:

Since the desktop is turned on and off a few times a day on different times, we need a different solution.
The solution is as follows: when the desktop gets turned off, it will make its daily backup. After making this backup it will sync both /backup/server and /backup/desktop. If the daily backup is allready made, meaning the desktop is not turned off for the first time that day, no further action will be taken.

For the desktop I wrote a tiny script, put it in /usr/local/sbin/backup
Code:

#!/bin/bash
if [ -a /backup/desktop/etc-desktop-`date +%d-%m-%Y`.tar.gz ]
then
        exit
else
        tar -czpf /backup/desktop/etc-desktop-`date +%d-%m-%Y`.tar.gz /etc && \
        chmod 600 /backup/desktop/etc-desktop-`date +%d-%m-%Y`.tar.gz && \
        rsync -a /backup/desktop/* rsync://192.168.0.1/desktop-backup && \
        rsync -a rsync://192.168.0.1/server-backup /backup/server
fi

Don't forget to make it executable:
Code:
chmod 700 /usr/local/sbin/backup

First the script checks wheter the daily backup is allready made or not. If it is made, there's just an exit. If it isn't made, there's four things to do:
- First the tarring, making a tar of /etc
- Second, the backup is chmodded 700, this is useful if you have other users on your server. The backup will be stored as user root (from the rsyncd.conf) and with rwx permissions for root only.
- The third step is rsyncing the desktop backup, this will transfer our new tar.
- The final step retrieves the backup from the server to store on your desktop.

If you chose not to make daily tars, the scritpt will be useless and all you need is:
Code:
rsync -rptgoL /backup/desktop/* rsync://192.168.0.1/desktop-backup && \
rsync -rptgoL rsync://192.168.0.1/server-backup /backup/server

-L will make sure the symlinks are being followed.

Last, we have to start the script every time the deskto shuts down, do it by adding this line:
Code:
/usr/local/sbin/backup
to /etc/conf.d/local.stop


That was all!
I hope it's clear and useful!
If you have any comments and/or improvements, please post them.


Please note this setup is designed for a local network with rsyncd not accessible from the internet!! Use your firewall!!

Edit:
07/06/04: Created a little script for the server's cronjob


Last edited by BoZ on Mon Jun 07, 2004 8:25 pm; edited 2 times in total
Back to top
View user's profile Send private message
BlackDuck
n00b
n00b


Joined: 28 Jun 2003
Posts: 67
Location: Canberra.au

PostPosted: Fri Jun 04, 2004 11:55 am    Post subject: Reply with quote

Hey, thanks for this really good how-to. I've always wondered how to put together an automatic backup system for my setup. This how-to is really educational and practical. This would be really useful as part of the official installation documentation... perhaps in a post-installation section.
_________________

Join the unanswered post initiative
Dell GX1 PIII 500 512MB Onboard ATI Rage Pro 3D | HP P1120 21" 1600x1200@85Hz
2 HP Vectra VE PIII 500 512MB Onboard Matrox G200
Back to top
View user's profile Send private message
Sfynx
n00b
n00b


Joined: 01 Jun 2002
Posts: 50

PostPosted: Fri Jun 04, 2004 3:52 pm    Post subject: Reply with quote

If you want to make incremental backups (useful when backing up a big amount of data every day that is not changing much), I'd suggest using rdiff-backup. That's a python script that uses rsync to backup to a directory and automatically takes care of increments (you can restore any previous instance or delete old increments and so on). This way, you can keep a lot of backups while not wasting disk space, because it only stores the changes. It can login though SSH to get the files, so you don't have to run a rsyncd. rdiff-backup is available in Portage.

btw: I would not run a rsyncd, but let a script copy your /etc files to another directory periodically, where a normal user can access them (say you make an extra user backup). Then you can let rsync get the files through SSH with public key authentication, instead of having a non-chrooted rsyncd open (that had some security problems in the past). If you restrict for one IP only this is not very relevant of course ;)
_________________
I'm the great Cornholio!
Are you threatening me?
Back to top
View user's profile Send private message
Braempje
l33t
l33t


Joined: 31 Jan 2003
Posts: 748

PostPosted: Sat Jun 05, 2004 9:15 am    Post subject: Reply with quote

What's wrong with unison? That does just what you want, right?
_________________
Dictionary of the Flemish Sign Language - Woordenboek Vlaamse Gebarentaal
Back to top
View user's profile Send private message
BoZ
Tux's lil' helper
Tux's lil' helper


Joined: 27 Aug 2003
Posts: 114
Location: The Netherlands

PostPosted: Sat Jun 05, 2004 10:28 am    Post subject: Reply with quote

I'm sure unison can do the same, but that's not the reason I made this.
First of all, if I need a script/program like this and I have time for it, I like to make it myself. That way I know exactly what it does, why, when and how to change it. After I made it, why not share it with everybody? I don't expect everybody on the forum to use it, but some people might find it useful. If you prefer unison or any other program that can do the same, just use it.
Back to top
View user's profile Send private message
andrewy
l33t
l33t


Joined: 07 Apr 2004
Posts: 602

PostPosted: Sat Jun 05, 2004 1:08 pm    Post subject: Reply with quote

Thanks, this will be really useful once I get around to setting up automated backups.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum