Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
DAR weekly/monthly system backup bash script
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
sven
Apprentice
Apprentice


Joined: 19 Apr 2002
Posts: 274

PostPosted: Sun Nov 30, 2003 11:41 am    Post subject: DAR weekly/monthly system backup bash script Reply with quote

DAR weekly/monthly system backup bash script

Hello folks!

I've created a tiny but useful bash script which, when put into /etc/cron.weekly, will create a full system backup every month and a differential backup every week. The differential backups will be overwritten each week so there's only one full and one up-to-date differential backup per month. The script uses DAR. There is an (masked) ebuild for it app-arch/dar. What is DAR? Here's a snipplet from the official website at http://dar.linux.free.fr/:

Quote:
dar is a shell command, that makes backup of a directory tree and files. It is released under the GNU General Public License (GPL in the following) and actually has been tested under Linux, Windows and Solaris. Since version 2.0.0 an Application Interface (API) is available to open the way to external independent Graphical User Interfaces (GUIs). An extension of this API (in its version 2) is in the air, for release 2.1.0, and would overcome some limitation of API version 1. This API relies on the libdar library which is the core part of DAR programs and, as such, is released under the GPL. In consequences, to use it, your program must be released under the GPL, no commercial use will be tolerated.

Here's the script:

Code:
#!/bin/sh
#
# This script creates a full backup every month
# and a differential backup every week using DAR
# (http://dar.linux.free.fr/) so DAR needs to
# be installed (ebuild app-arch/dar on Gentoo
# Linux).
#
# Please adjust the settings to your needs,
# then copy this script into /etc/cron.weekly
# and chmod 744 it.
#
# Version 1.0.2
# 2003/11/30
#
# Sven Jacobs (sven.jacobs@web.de)
# www.sven-jacobs.de
#

NICE="/bin/nice -19"
DAR="/usr/bin/dar"
BACKUP_DEST="/root/backup/"
DATESTRING=`date +%b-%Y | tr 'A-Z' 'a-z'`
FILENAME_PREFIX="gentoo-system-"
FILENAME=$FILENAME_PREFIX$DATESTRING
FILENAME_DIFF=$FILENAME-diff
OPTIONS="-R /
         -P proc
         -P sys
         -P tmp
         -P var/tmp/portage
         -P mnt/.init.d
         -P mnt/cdrom
         -P mnt/floppy
         -P usr/portage/distfiles
         -P root/.ccache
         -P home/p2p/.mldonkey/temp
         -P home/p2p/.mldonkey/incoming
         -X $FILENAME.*.dar
         -X $FILENAME_DIFF.*.dar
         -D -y9 -s 700M -w"
PRE_ACTION="/bin/mount /boot"
POST_ACTION="/bin/umount /boot"

$PRE_ACTION
if [[ -a $BACKUP_DEST$FILENAME.1.dar ]]; then
  $NICE $DAR -c $BACKUP_DEST$FILENAME_DIFF -A $BACKUP_DEST$FILENAME $OPTIONS
else
  $NICE $DAR -c $BACKUP_DEST$FILENAME $OPTIONS
fi
$POST_ACTION

Name it system-backup, chmod 744 it and put it into /etc/cron.weekly. The script places the backups into /root/backup so create that dir or adjust the script.

Have fun!


Last edited by sven on Sun Nov 30, 2003 12:37 pm; edited 2 times in total
Back to top
View user's profile Send private message
x10n
n00b
n00b


Joined: 30 Nov 2003
Posts: 1

PostPosted: Sun Nov 30, 2003 12:32 pm    Post subject: Reply with quote

That's a very useful tiny helper. No more reading dar's manpage for making full system backups.


Got fun! ;)
Back to top
View user's profile Send private message
revertex_dot_net
n00b
n00b


Joined: 14 Jul 2003
Posts: 39

PostPosted: Sun Dec 14, 2003 1:49 pm    Post subject: Reply with quote

Dar seems a perfect backup tool for home workstations/standalone machines, thanks for nice script.
Just in case, if someone needs a tool to recover a Dar backup from a unbootable machine, this live-cd seems realy usefull.
http://www.systemrescuecd.org/
(based on Gentoo LiveCd, tons of system tools, graphical qtparted, graphical links, a must have!)
Back to top
View user's profile Send private message
TheCoop
Veteran
Veteran


Joined: 15 Jun 2002
Posts: 1814
Location: Where you least expect it

PostPosted: Sun Dec 14, 2003 6:08 pm    Post subject: Reply with quote

how do you change the script so it does diff backups each day and a full backup each week?
_________________
95% of all computer errors occur between chair and keyboard (TM)

"One World, One web, One program" - Microsoft Promo ad.
"Ein Volk, Ein Reich, Ein Führer" - Adolf Hitler

Change the world - move a rock
Back to top
View user's profile Send private message
nsahoo
l33t
l33t


Joined: 17 Jul 2003
Posts: 618

PostPosted: Sun Dec 14, 2003 11:10 pm    Post subject: Reply with quote

I'd suggest you take a look at flexbackup too. It's in the portage tree.
Back to top
View user's profile Send private message
Simon196405
n00b
n00b


Joined: 22 Sep 2003
Posts: 42
Location: Utrecht, Holland

PostPosted: Tue Dec 07, 2004 8:47 pm    Post subject: Reply with quote

I use this in /etc/darrc

Using dar this way calling dar is much more simple. The only option needed is the -c option.
Also restore gets more easy because there is no need to use the -R option anymore.

I think the file is commented well. If anybody has some ideas about making it more complete I would like to know them.
Together we should be able to figure out the perfect configuration file for dar.

Code:

## This is the configuration file for dar.

default:
 -h

all:
# start at root
 -R /

create:
# Exclude from backup
     # skip only contents of directories not the directories itself
        -D
     # these directories
        -P proc
        -P sys
        -P dev
        -P tmp
        -P var/tmp
        -P var/cache
        -P usr/portage/distfiles
        -P srv/backup
        -P "home/*/Desktop/Trash"
        -P root/Desktop/Trash
     # these filetypes
        -X "*~" -X ".*~"
        -X "*.dar"
        -X "*.iso"
#  don't try to compress already compressed files
     # sound/music
        -Z *.mp3
        -Z *.ogg
     # pictures icons
        -Z *.jpg -Z *.JPG
        -Z *.png
     # movies
     
     # documents
        -Z *.pdf
 # openoffice
        -Z *.sx? -Z.st?
 # koffice
        -Z *.kwd -Z *.kwt -Z *.ksp
 # OASIS
 
     # general
        -Z *.gz
        -Z *.bz2
        -Z *.zip
# miscelenious options
     # compress with bzip2
        -y
     # use 700M slices (CD's)
        -s 700M
        -E "mkisofs -r -quiet -o %p/%b.%n.iso %p/%b.%n.dar"
        -E "rm %p/%b.%n.dar"


Simon Siemonsma
Back to top
View user's profile Send private message
bludger
Guru
Guru


Joined: 09 Apr 2003
Posts: 389

PostPosted: Wed Dec 08, 2004 2:49 pm    Post subject: Reply with quote

This looks great. I will certainly try it out. Does anyone also have any experience using parchive in association with dar?
Back to top
View user's profile Send private message
bludger
Guru
Guru


Joined: 09 Apr 2003
Posts: 389

PostPosted: Mon Jan 03, 2005 10:54 am    Post subject: Reply with quote

I recently tried dar out. I found that when encryption was used, my cpu utilisation was sky high and the full system backup took many hours (like around 10). When I turned off encryption, it backed up the entire system in about 1 hour, which is more reasonable. The process needed about 40% of my 500MB RAM, which is very high but could be lived with. I have an Athlon 1000MHz.

I then tried to do a differential backup after a small number of files had changed. This process then chewed up over 75% of my RAM, which started causing other processes to be killed. This is listed in the limitations section of the manual but without any workarounds that I could find.

Has anyone else had any similar experiences? Are there any possible workarounds? By the look of things, if you want to do differential backups of a large system, you need a database based file catalogue. I think I will now have to have a look at Arkeia or Bacula.
Back to top
View user's profile Send private message
Greven
Tux's lil' helper
Tux's lil' helper


Joined: 28 Jul 2002
Posts: 138

PostPosted: Mon Jan 03, 2005 3:50 pm    Post subject: Reply with quote

I will have to try this out...... I have a server I need to backup at work, and this maybe my care.
_________________
veritas vos liberabit...
Linux User Number: 346805
Wine-Wiki
AMD 64 3500+ | MSI "K8T NEO2-FIR" | mushkin Dual Channel DDR 400
Back to top
View user's profile Send private message
finr
Tux's lil' helper
Tux's lil' helper


Joined: 24 Feb 2004
Posts: 126

PostPosted: Wed Jan 05, 2005 9:22 am    Post subject: Reply with quote

revertex_dot_net wrote:
Dar seems a perfect backup tool for home workstations/standalone machines, thanks for nice script.
Just in case, if someone needs a tool to recover a Dar backup from a unbootable machine, this live-cd seems realy usefull.
http://www.systemrescuecd.org/
(based on Gentoo LiveCd, tons of system tools, graphical qtparted, graphical links, a must have!)


Thank you for the tip, but I guess you meant http://www.sysresccd.org/?
Back to top
View user's profile Send private message
Massimo B.
Veteran
Veteran


Joined: 09 Feb 2005
Posts: 1539
Location: Germany

PostPosted: Wed Nov 01, 2006 3:29 pm    Post subject: Reply with quote

Ok, the advantage over tar is providing random access on compressed or even encrypted archives. Making slices that fit on CD/DVD makes it ready for these medium.
But can you change that script to write slices directly to the medium by using cdrdao or others? Maybe by --execute which would unfortunately still save to harddrive and copy over to cd afterwards. Would it be possible directly writing to these mediums?
_________________
ppc:PowerBook5,8 15"(1440)-G4/1.67,2G|amd64:HP EliteBook 8560w,i7-2620M,16G|amd64:Acer Z5610 (Core2QuadQ8200),8G|amd64-prefix:OpenSuse
Lila-Theme
Back to top
View user's profile Send private message
Massimo B.
Veteran
Veteran


Joined: 09 Feb 2005
Posts: 1539
Location: Germany

PostPosted: Sat Dec 23, 2006 12:09 pm    Post subject: Reply with quote

I'd like to provide you also my script that I made for encrypted home backups:
Code:
# /usr/local/bin/1darhome.sh
#!/bin/bash
# ---------------------------------------------
NICE="8"

FSROOT="/home/pao"
DST="/mnt/firewire/backup"

BASE_NAME="backup.home"
BACKUP_FILENAME="$BASE_NAME.`date '+%Y_%m'`"
DIFF_FILENAME="$BASE_NAME.diff.`date '+%Y_%m_%d'`"
CAT_FILENAME="$BASE_NAME.cat.last"
CAT_NEW_FILENAME="$BASE_NAME.cat.new"

BACKUP_DST="$DST/$BACKUP_FILENAME"
DIFF_DST="$DST/$DIFF_FILENAME"
CAT_DST="$DST/$CAT_FILENAME"
CAT_NEW_DST="$DST/$CAT_NEW_FILENAME"

EXCLUDE_DIRS="Daten Desktop* download mnt Rucksack smb4k *trash Trash .evolution/mail .evolution/cache .local/share/Trash .thumbnails .pan/dfn"
EXCLUDE_FILES=""

EXCLUDE_COMPRESS="*.jpg *.JPG *.bz2 *.zip *.gz *.png *.mp* *.gif"
CYPHER="bf:"
OPTS="-s 3800M -p -b -y -v -J $CYPHER -K $CYPHER"
# --------------------------------------------
EXCLUDE_C="-Z '${EXCLUDE_COMPRESS// /' -Z '}'"
EXCLUDE_D="-P '${EXCLUDE_DIRS// /' -P '}'"
EXCLUDE_F="-X '${EXCLUDE_FILES// /' -X '}'"

echo "### DAR Backup script, starting "$BASE_NAME" ###"$'\n'
echo "### Current settings:"$'\n'
echo "NICE            =$NICE"
echo "BACKUP_DST      =$BACKUP_DST"
echo "DIFF_DST        =$DIFF_DST"
echo "CAT_DST         =$CAT_DST"
echo "LST_CAT_DST     =$LST_CAT_DST"
echo "OPTS            =$OPTS"
echo "FSROOT          =$FSROOT"
echo "EXCLUDE_COMPRESS=$EXCLUDE_C"
echo "EXCLUDE_DIRS    =$EXCLUDE_D"
echo "EXCLUDE_FILES   =$EXCLUDE_F"
echo "###"$'\n'

renice  $NICE $$ >/dev/null
cd $DST
if [[ -a "$BACKUP_DST.1.dar" ]]; then
        echo "Catalog found, continuing with differential backup..."
        eval    dar  -c $DIFF_DST  -R $FSROOT  -A $CAT_DST  -G $CAT_NEW_DST  $OPTS  $EXCLUDE_C $EXCLUDE_D $EXCLUDE_F &&\
                # encrypting the catalog and deleting the unecrypted:
#               dar  -K bf:  --isolate $CAT_DST-encr  -A $CAT_DST-new &&\
                rm -vf $CAT_DST.1.dar &&\
                mv -vf $CAT_NEW_DST.1.dar $CAT_DST.1.dar
else
        echo "No catalog found, initiating complete backup..."
        eval    dar  -c $BACKUP_DST  -R $FSROOT  -G $CAT_NEW_FILENAME  $OPTS  $EXCLUDE_C $EXCLUDE_D $EXCLUDE_F &&\
#               dar  -K bf:  --isolate $CAT_DST-encr  -A $CAT_DST-new &&\
                rm -vf $CAT_DST.1.dar &&\
                mv -vf $CAT_NEW_DST.1.dar $CAT_DST.1.dar
fi

_________________
ppc:PowerBook5,8 15"(1440)-G4/1.67,2G|amd64:HP EliteBook 8560w,i7-2620M,16G|amd64:Acer Z5610 (Core2QuadQ8200),8G|amd64-prefix:OpenSuse
Lila-Theme
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum