Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
a great backup system for gentoo workstations
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3, 4  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
hadfield
Retired Dev
Retired Dev


Joined: 18 Mar 2003
Posts: 308
Location: Vancouver, BC, Canada

PostPosted: Wed Nov 05, 2003 5:33 am    Post subject: Reply with quote

Right, but by the time the script gets to the line that does the 'ls' on your backup directory (i.e. the line that's generating the error) there SHOULD be at least one file there. This is because the line of code that creates the tar file in that directory has already been executed. This is why I think there's something wrong with the generation of the tar file. Have any tgz files been created in your backup directory? Your CURRENT backup should be there (i.e. the backup you're trying to create).
Back to top
View user's profile Send private message
odegard
Guru
Guru


Joined: 08 Mar 2003
Posts: 324
Location: Trondheim, NO

PostPosted: Wed Nov 05, 2003 6:29 am    Post subject: Reply with quote

Maybe I don't understand what you're saying but there are only one file in the backup dir. manual tar works just fine, this is how I tested it:

Code:

touch list
echo file > list
cat list | xargs tar zlcf archive


and I got an archive containing the file "file".

Could it be something with permissions? Can you give me the exact syntax I need to make the backup.cron executable?

Thanks for your time!
Back to top
View user's profile Send private message
hadfield
Retired Dev
Retired Dev


Joined: 18 Mar 2003
Posts: 308
Location: Vancouver, BC, Canada

PostPosted: Wed Nov 05, 2003 8:50 am    Post subject: Reply with quote

Yeah, I often have trouble explaining things. :/ Anyway, If you put a file of the form:
backup.2003-10-30-2339.tgz
into your backup folder, do you still get the error message? Does your backup file get successfully created?
Back to top
View user's profile Send private message
odegard
Guru
Guru


Joined: 08 Mar 2003
Posts: 324
Location: Trondheim, NO

PostPosted: Wed Nov 05, 2003 9:01 am    Post subject: Reply with quote

If I put an empty file with that name in my .backup directory and run the script nothing happens. No error message and the the file stays with the name I gave it and with 0 bytes.

The weird thing is that the script *worked* before I reinstalled.
To make the script executable, I do
Code:

chmod u+x backup.cron

Correct?
Back to top
View user's profile Send private message
razamatan
Apprentice
Apprentice


Joined: 28 Feb 2003
Posts: 160

PostPosted: Thu Nov 06, 2003 3:15 am    Post subject: Reply with quote

hmm... first off... i'd

Code:
chmod 700 backup.cron


just so root is the only one who can run it...

now... as for the problems...

you should have these things existing:

Code:
- the directory /home/.backup
- a file /home/.backup/sys.lst


that's about it... and that lst file should list your directories and such.
_________________
a razamatan doth speaketh,
"Never attribute to malice, that which can be adequately explained by stupidity"
Back to top
View user's profile Send private message
razamatan
Apprentice
Apprentice


Joined: 28 Feb 2003
Posts: 160

PostPosted: Thu Nov 06, 2003 3:22 am    Post subject: Reply with quote

in general... if a script is giving you troubles, run it line by line (cut and paste)... see where it borks..

the thing is... this script should work... try renaming your list to sys.lst since it contains /boot (and assuming that you need to mount it)...
_________________
a razamatan doth speaketh,
"Never attribute to malice, that which can be adequately explained by stupidity"
Back to top
View user's profile Send private message
odegard
Guru
Guru


Joined: 08 Mar 2003
Posts: 324
Location: Trondheim, NO

PostPosted: Thu Nov 06, 2003 3:49 am    Post subject: Reply with quote

Ahh...that was the error, my list file was called backup.lst and not sys.lst. It works now. Thank you very much!
Back to top
View user's profile Send private message
razamatan
Apprentice
Apprentice


Joined: 28 Feb 2003
Posts: 160

PostPosted: Thu Nov 06, 2003 9:16 am    Post subject: Reply with quote

odegard wrote:
Ahh...that was the error, my list file was called backup.lst and not sys.lst. It works now. Thank you very much!


honestly... it shouldn't matter...

try naming your list back to backup.lst (still under /home/.backup)... then in the backup.cron script, do this:

Code:
BOOT=backup


essentially, you should set BOOT to be the list that contains the /boot directory, so the script can properly (u)mount the thing.

but still... even if it wasn't properly mounting boot... it would archive the directory...

to really test to see what was wrong, revert everything back to when you were getting errors and

change
Code:
                ${BKPDIR}/${type}.`date +%Y-%m-%d-%H%M`.${ext} > /dev/null 2>&1

to
Code:
                ${BKPDIR}/${type}.`date +%Y-%m-%d-%H%M`.${ext}


(this just removes the redirection of errors and warnings for tar)
_________________
a razamatan doth speaketh,
"Never attribute to malice, that which can be adequately explained by stupidity"
Back to top
View user's profile Send private message
odegard
Guru
Guru


Joined: 08 Mar 2003
Posts: 324
Location: Trondheim, NO

PostPosted: Fri Nov 07, 2003 5:58 am    Post subject: Reply with quote

razamatan wrote:

honestly... it shouldn't matter...

try naming your list back to backup.lst (still under /home/.backup)... then in the backup.cron script, do this:

Code:
BOOT=backup



Thats what I did.

razamatan wrote:

to really test to see what was wrong, revert everything back to when you were getting errors and

change
Code:
                ${BKPDIR}/${type}.`date +%Y-%m-%d-%H%M`.${ext} > /dev/null 2>&1

to
Code:
                ${BKPDIR}/${type}.`date +%Y-%m-%d-%H%M`.${ext}


(this just removes the redirection of errors and warnings for tar)


Ok, I left the script in working order, that is, I did not change back the name of the list to 'sys' in the script, and did this:
Code:

#               ${BKPDIR}/${type}.`date +%Y-%m-%d-%H%M`.${ext} > /dev/null 2>&1
                ${BKPDIR}/${type}.`date +%Y-%m-%d-%H%M`.${ext}


And I get this error msg:

Code:

06:49:44 root /etc/cron.daily # ./backup.cron
tar: Removing leading `/' from member names
tar (child): /boot: Cannot open: Is a directory
tar (child): Error is not recoverable: exiting now
xargs: tar: terminated by signal 13
./backup.cron: line 40: /home/.backup/backup.2003-11-07-0649.tgz: No such file or directory


Is there something wrong with my tar-tools? Another thing thats funny is that I can't enter tar archives when using krusader...it says protocol not supported but I can pack/unpack just fine...just not browse the archive...
Back to top
View user's profile Send private message
razamatan
Apprentice
Apprentice


Joined: 28 Feb 2003
Posts: 160

PostPosted: Fri Nov 07, 2003 5:17 pm    Post subject: Reply with quote

how does /boot look on your machine...

is it a separate partition that is usually left unmounted?
_________________
a razamatan doth speaketh,
"Never attribute to malice, that which can be adequately explained by stupidity"
Back to top
View user's profile Send private message
odegard
Guru
Guru


Joined: 08 Mar 2003
Posts: 324
Location: Trondheim, NO

PostPosted: Sat Nov 08, 2003 8:31 pm    Post subject: Reply with quote

Yes.
Back to top
View user's profile Send private message
odegard
Guru
Guru


Joined: 08 Mar 2003
Posts: 324
Location: Trondheim, NO

PostPosted: Sat Nov 08, 2003 8:32 pm    Post subject: Reply with quote

Yes.
Back to top
View user's profile Send private message
odegard
Guru
Guru


Joined: 08 Mar 2003
Posts: 324
Location: Trondheim, NO

PostPosted: Sat Nov 08, 2003 8:33 pm    Post subject: Reply with quote

Yes.
Back to top
View user's profile Send private message
speedster
n00b
n00b


Joined: 31 Oct 2003
Posts: 39
Location: North Carolina, USA

PostPosted: Thu Dec 04, 2003 3:28 pm    Post subject: easiest document backup Reply with quote

I know this is much more elementary; but for backing up my Documents (which change daily) I use the following in cron (I use fcron to run it):

15,45 * * * * cp -ruf /home/SECRET/Documents /mnt/DESTINATION

every 15 and 45 past the hour, documents that have changed are automatically copied to my destination partition. Of course, no compression is used, but who cares with a 100GB hard drive? This hardly uses any system resources as well, except the first time it runs. The same thing could be used for any system files or whatever.
_________________
Gentoo is highly addictive....
Back to top
View user's profile Send private message
my_nick
n00b
n00b


Joined: 07 Jul 2003
Posts: 28
Location: Slovenia

PostPosted: Thu Dec 25, 2003 10:46 am    Post subject: Reply with quote

One tip from me:

Exclude mldonkey/xmule/whatever temp files from the backup because you can't extract the archive containing these files.

Been there, done that, had to reinstall :cry:
Back to top
View user's profile Send private message
mkamonster
n00b
n00b


Joined: 06 Aug 2003
Posts: 51
Location: The Hague, Netherlands

PostPosted: Wed Dec 31, 2003 10:08 am    Post subject: Reply with quote

Are incremental backups possible with this script? I like the incremental backup's a lot.
Back to top
View user's profile Send private message
razamatan
Apprentice
Apprentice


Joined: 28 Feb 2003
Posts: 160

PostPosted: Tue Jan 06, 2004 1:49 am    Post subject: Reply with quote

mkamonster wrote:
Are incremental backups possible with this script? I like the incremental backup's a lot.


never really looked into that... i personally just wanted weekly full backups.

i'm sure you can add the incremental stuff, but it'll add more lines of code. :)
_________________
a razamatan doth speaketh,
"Never attribute to malice, that which can be adequately explained by stupidity"
Back to top
View user's profile Send private message
Starfox
Tux's lil' helper
Tux's lil' helper


Joined: 04 Sep 2002
Posts: 93

PostPosted: Thu Feb 05, 2004 9:24 pm    Post subject: Reply with quote

HI folks,

as i wanted to process only some backups every day i just wrote a small patch to razas script to take the first command line argument (if there is one!) as backuplist.

Just patch the original script with the following lines:
Code:
26,35c26
< # command line patch added by starfox
< # usage (e.g. if you have test.lst in your $BKPDIR):
< #     ./backup.sh test
< # now only the "test.lst" file is processed for backup
< if [ -f ${BKPDIR}/${1}.lst ]; then
<       lists=${BKPDIR}/${1}.lst
< else
<       lists=${BKPDIR}/*.lst
< fi
< # end command line patch
---
> lists=${BKPDIR}/*.lst


enjoy
Back to top
View user's profile Send private message
timaios
n00b
n00b


Joined: 20 Mar 2004
Posts: 14

PostPosted: Sat Mar 20, 2004 4:49 pm    Post subject: Reply with quote

gatiba wrote:
Great script !
Would be nice if it could split the backup file in more parts if it's > 700mb for example (for burning ;) )


if you haven't still found a way to split your backup-files
in order to fit the in cds, you can also check split
"split -b 700m <tarfile> <prefix>"
Back to top
View user's profile Send private message
Guglie
Apprentice
Apprentice


Joined: 01 Mar 2004
Posts: 240
Location: switzerland - zürich

PostPosted: Wed Sep 21, 2005 7:20 pm    Post subject: Reply with quote

I've modified the script from razamatan with some new features:
- I use separate directories for the lists (~/.backup) and for the backups (NFS directory of my server)
- I make backups every week and every month: I will also have every time backups of the lasts 6 months and backups of the lasts 4 weeks (1 month)
this is possible running the script with
Code:
backup_script -t [monthly/weekly]

I've also added the -s list_name option, to select the list and the -l option to show a list of the availables lists

I copy here the full script and not a patch, because my code is much different than the code of razamatan
Code:
#!/bin/bash
# file: backup_script.sh
# date: september 2004
# original author: razamatan
# version: 0.2
# license: GPL-2

BKPDIR=/mnt/backup/galvorn     # directory where the backups go
LISTDIR=/home/evellon/.backup  # directory where the lists are
BOOT=sys                       # list that has /boot
NUMBKPS_weekly=4               # how many backups to keep in weekly-dir
NUMBKPS_monthly=6              # how many backups to keep in monthly-dir
EXT=tar.gz                     # extension of backups

function whelp
{
   echo "Usage: $(basename ${0})"
   echo
   echo "Options:"
   echo " -s list   backup only the directories of the specified list"
   echo " -t time   weekly or monthly"
   echo " -l        show the list of the backup-modes availables"
   echo " -h        watch this help and exit"
   exit 0
}

function wlist
{
   echo "Selectable lists:"
   for file in `ls ${LISTDIR}/*.lst`; do
      echo $(basename $file .lst)
   done
   exit 0
}

function make_backup
{
   if [ -z "$lists" ] ; then lists=${LISTDIR}/*.lst; fi
   if [ -z "$time" ] ; then time="weekly"; fi
   
   BKPDIR=$BKPDIR/$time
   if [ ! -d "$BKPDIR" ] ; then mkdir $BKPDIR; fi

   eval num=\$"NUMBKPS_"$time
   
   for list in `ls ${lists}`; do
      type=`basename ${list} .lst`
      
      if [ ${type} = ${BOOT} ] ; then mount /boot; fi
      
      echo "backupping "$type" ..."
      cat ${list} | xargs tar zlcf ${BKPDIR}/${type}.`date +%Y-%m-%d-%H%M`.${EXT} > /dev/null 2>&1      
      
      if [ ${type} = ${BOOT} ] ; then umount /boot; fi
      
      for evict in `ls -t ${BKPDIR}/${type}.*.${EXT}`; do
         if [ ${num} -le 0 ] ; then
            rm -f ${evict}
         else
            num=$((${num}-1))
         fi
      done
   done
   exit 0
}

if [ ! -d ${LISTDIR} ] ; then
   echo "The LISTDIR ${LISTDIR} is not a valid directory or is non-existant"
elif [ ! -d ${BKDIR} ] ; then
   echo "The BKDIR ${BKDIR} is not a valid directory or is non-existant"
fi

while getopts ":hlt:s:" OPTION; do
   case $OPTION in
      h) whelp ;;
      l) wlist ;;
      t)   if [ "$OPTARG" = "weekly" ] || [ "$OPTARG" = "monthly" ] ; then
            time=$OPTARG
         else
            echo "Time mus be weekly or monthly"
            exit 0
         fi
         ;;
      s)
         if [ -f ${LISTDIR}/$OPTARG.lst ]; then
            lists=${LISTDIR}/$OPTARG.lst
         else
            echo "The list $OPTARG is not available"
            echo
            wlist
         fi
         ;;
      *) whelp ;;
   esac
done

make_backup


EDIT: I have patched the original code with the suggestions of burndive
_________________
A! Elbereth Gilthoniel! silivren penna míriel o menel aglar elenath!


Last edited by Guglie on Sun Sep 25, 2005 3:02 pm; edited 2 times in total
Back to top
View user's profile Send private message
burndive
n00b
n00b


Joined: 21 Apr 2005
Posts: 5
Location: WA, USA

PostPosted: Sun Sep 25, 2005 6:10 am    Post subject: One minor big and a caveat. Reply with quote

Change BKPDIR to LISTDIR in the first line of this function (as shown below):
Also, ${BKPDIR}/weekly and ${BKPDIR}/monthly must already exist for the script to work.
Code:

function make_backup
{
   if [ -z "$lists" ] ; then lists=${LISTDIR}/*.lst; fi
   if [ -z "$time" ] ; then time="weekly"; fi

   ...
}
Back to top
View user's profile Send private message
Guglie
Apprentice
Apprentice


Joined: 01 Mar 2004
Posts: 240
Location: switzerland - zürich

PostPosted: Sun Sep 25, 2005 3:02 pm    Post subject: Re: One minor big and a caveat. Reply with quote

burndive wrote:
Change BKPDIR to LISTDIR in the first line of this function (as shown below):
Also, ${BKPDIR}/weekly and ${BKPDIR}/monthly must already exist for the script to work.

thanks for the testing: I have corrected this mistakes, this is the patch
Code:
--- backup_script-0.1.sh   2005-09-25 16:29:55.397696250 +0200
+++ backup_script-0.2.sh   2005-09-25 16:31:00.089739250 +0200
@@ -2,9 +2,10 @@
 # file: backup_script.sh
 # date: september 2004
 # original author: razamatan
+# version: 0.2
 # license: GPL-2
 
-BKPDIR=/mnt/backup             # directory where the backups go
+BKPDIR=/mnt/backup/galvorn     # directory where the backups go
 LISTDIR=/home/evellon/.backup  # directory where the lists are
 BOOT=sys                       # list that has /boot
 NUMBKPS_weekly=4               # how many backups to keep in weekly-dir
@@ -27,7 +28,6 @@
 {
    echo "Selectable lists:"
    for file in `ls ${LISTDIR}/*.lst`; do
-      #echo $file | sed -e 's/'$LISTDIR'/ /g'
       echo $(basename $file .lst)
    done
    exit 0
@@ -35,11 +35,13 @@
 
 function make_backup
 {
-   if [ -z "$lists" ] ; then lists=${BKPDIR}/*.lst; fi
+   if [ -z "$lists" ] ; then lists=${LISTDIR}/*.lst; fi
    if [ -z "$time" ] ; then time="weekly"; fi
    
    BKPDIR=$BKPDIR/$time
-   num=$( eval echo "\$NUMBKPS_${time}" )
+   if [ ! -d "$BKPDIR" ] ; then mkdir $BKPDIR; fi
+
+   eval num=\$"NUMBKPS_"$time
    
    for list in `ls ${lists}`; do
       type=`basename ${list} .lst`
@@ -68,8 +70,7 @@
    echo "The BKDIR ${BKDIR} is not a valid directory or is non-existant"
 fi
 
-while getopts ":hlt:s:" OPTION;
-do
+while getopts ":hlt:s:" OPTION; do
    case $OPTION in
       h) whelp ;;
       l) wlist ;;
@@ -94,3 +95,4 @@
 done
 
 make_backup
+

NB: I have modified the code in the previous post with this patch
_________________
A! Elbereth Gilthoniel! silivren penna míriel o menel aglar elenath!
Back to top
View user's profile Send private message
spindle
Apprentice
Apprentice


Joined: 01 Dec 2003
Posts: 243

PostPosted: Sat Oct 29, 2005 1:51 pm    Post subject: Reply with quote

This script is so great. Simple and straight-forward. Just what I was looking for.

I've been using it for a few weeks now. Works wonderfully.

Thanks!
Back to top
View user's profile Send private message
Orange Crush
n00b
n00b


Joined: 22 May 2006
Posts: 1

PostPosted: Mon May 22, 2006 1:59 pm    Post subject: Differentials Reply with quote

There's a pretty simple way of getting this script to do differential backups (i.e. every updated file since the last full backup so that you can restore with just the most recent full backup and the most recent differential.)

Just have two copies of the script . . .say "full.cron" and "differential.cron"

In full.cron, at some point before it runs tar add the line:

date > /var/lastbak

(that will record the date and time of the full backup)

Then in differential.cron just add "--newer /var/lastbak" to the tar options.

That should do it. ;)

-Nick
Back to top
View user's profile Send private message
micr0c0sm
Tux's lil' helper
Tux's lil' helper


Joined: 29 Oct 2005
Posts: 148
Location: New York

PostPosted: Sat May 27, 2006 8:52 pm    Post subject: Reply with quote

This would make a great wiki entry...
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page Previous  1, 2, 3, 4  Next
Page 3 of 4

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum