Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
HOWTO: Easily make a full system backup (stage4)
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3 ... 13, 14, 15, 16, 17, 18  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
BlinkEye
Veteran
Veteran


Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums

PostPosted: Mon Dec 05, 2005 12:06 pm    Post subject: Reply with quote

i wished i'd receive a mail when this thread gets a new post - sometimes it simply doesn't work (same problem with my phpbb forum) ...

bingobob wrote:
This script is great. One thing though. I would like to run it each week from a cron script. How can i make it non-interactive?

read me

-valheru- wrote:
I too get errors trying to bz2 with a file over 2.5 gigs. File intergrity error.

gz is fine though.

interesting. you may really mail a bz2 users list with this issue.

andrewd18 wrote:
I'm getting an issue - it can't pass the integrity check...

Code:
StubbornAesthetics ~ # ./mkstage4.sh
...
sh: line 1:  2708 Broken pipe       
...


Suggestions? Same problem funkoolow had, but there was no answer posted to his inquiry.
~~ Andrew D.

there's a copy and paste error ( -> broken pipe ... ). try downloading it.
_________________
Easily backup up your system? klick
Get rid of SSH Brute Force Attempts / Script Kiddies klick
Back to top
View user's profile Send private message
andrewd18
Guru
Guru


Joined: 11 Apr 2004
Posts: 364
Location: Wisconsin, USA

PostPosted: Mon Dec 05, 2005 1:43 pm    Post subject: Reply with quote

Blink Eye wrote:
there's a copy and paste error ( -> broken pipe ... ). try downloading it.


I got the file from doing your wget command, so it might be something there. Come to think of it, my file was over 2.5 Gigs, so -valheru-'s error might apply to me as well. I'll try tar.gz and see how that goes.

~~ Andrew D.
_________________
Keep Your Toolchain Stable! - emwrap.sh

There's no place like ::1
Back to top
View user's profile Send private message
tgh
Apprentice
Apprentice


Joined: 05 Oct 2005
Posts: 222

PostPosted: Mon Dec 05, 2005 4:50 pm    Post subject: Reply with quote

bingobob wrote:
This script is great. One thing though. I would like to run it each week from a cron script. How can i make it non-interactive?

My guess would be to create a response file, with the keypresses that you need in order to answer the questions. Then pipe that response file into the script, something like:

mkstage4.sh < response.txt

The response.txt file might look like:

Code:
y
1


(I haven't tried this personally.)
Back to top
View user's profile Send private message
BlinkEye
Veteran
Veteran


Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums

PostPosted: Mon Dec 05, 2005 4:57 pm    Post subject: Reply with quote

this has been posted already and is explained it the wiki version (meaning: it works :wink: )
_________________
Easily backup up your system? klick
Get rid of SSH Brute Force Attempts / Script Kiddies klick
Back to top
View user's profile Send private message
satanskin
Guru
Guru


Joined: 25 Apr 2005
Posts: 353

PostPosted: Tue Dec 06, 2005 5:44 am    Post subject: Reply with quote

My files were split at a little under 4Gb as well. The bz file fails integrity check and the gz file passes with fly colors.
Back to top
View user's profile Send private message
carpman
Advocate
Advocate


Joined: 20 Jun 2002
Posts: 2202
Location: London - UK

PostPosted: Mon Dec 19, 2005 1:16 pm    Post subject: Reply with quote

Ok here is one that not sure if been discussed but woth mentioning. i did a full backup /home etc, thing is i had a dvd mounted and script did backup of /media/cdrecorder (this what my dvd/cdwriter is mounted as) could be an idea to add /media as a default excluded dir!
_________________
Work Station - 64bit
Gigabyte GA X48-DQ6 Core2duo E8400
8GB GSkill DDR2-1066
SATA Areca 1210 Raid
BFG OC2 8800 GTS 640mb
--------------------------------
Notebook
Samsung Q45 7100 4gb
Back to top
View user's profile Send private message
BlinkEye
Veteran
Veteran


Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums

PostPosted: Mon Dec 19, 2005 3:07 pm    Post subject: Reply with quote

ah, /media. i guess this is a new dir due to the lsb adaption. i'll add it to the default exclude dir, thanks.
_________________
Easily backup up your system? klick
Get rid of SSH Brute Force Attempts / Script Kiddies klick
Back to top
View user's profile Send private message
carpman
Advocate
Advocate


Joined: 20 Jun 2002
Posts: 2202
Location: London - UK

PostPosted: Mon Dec 19, 2005 7:26 pm    Post subject: Reply with quote

BlinkEye wrote:
ah, /media. i guess this is a new dir due to the lsb adaption. i'll add it to the default exclude dir, thanks.


pleased to help out. also don't forget /srv this is server related
_________________
Work Station - 64bit
Gigabyte GA X48-DQ6 Core2duo E8400
8GB GSkill DDR2-1066
SATA Areca 1210 Raid
BFG OC2 8800 GTS 640mb
--------------------------------
Notebook
Samsung Q45 7100 4gb
Back to top
View user's profile Send private message
allucid
Veteran
Veteran


Joined: 02 Nov 2002
Posts: 1314
Location: atlanta

PostPosted: Mon Dec 19, 2005 9:06 pm    Post subject: Reply with quote

Is it possible to combine this method with one of the 'Make you own Gentoo LiveCD' howtos? That way you have a rescue/recovery disk all in one. :)
Back to top
View user's profile Send private message
BlinkEye
Veteran
Veteran


Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums

PostPosted: Mon Dec 19, 2005 11:34 pm    Post subject: Reply with quote

yes of course. it has shortly been mentioned here (although i answered the question not quite correctly about the use of a livecd - i just haven't had the need for it as i do have the gentoo installation of my laptop cloned on a external usb harddisk (my stage4 test installation :wink: ) ).
_________________
Easily backup up your system? klick
Get rid of SSH Brute Force Attempts / Script Kiddies klick
Back to top
View user's profile Send private message
carpman
Advocate
Advocate


Joined: 20 Jun 2002
Posts: 2202
Location: London - UK

PostPosted: Tue Dec 20, 2005 9:22 am    Post subject: Reply with quote

Hello, i have a probelm though not sure if it is csript error or because of what i am trying, i have a working amd XP system and did a stage so as to copy install to a duron 1100 system, although different cpu it should work fine as can use same cflags, the early XP just being a duron with extra cache, thing is the XP i am using is mobile XP2500+.

Problem is that after restoring to new machine and when in chroot i cannot run:

Code:

nano (illegal operation)
or do
make menuconfig (complains about no ncurses-devel lib)
or do
make && make modules_install


emerge works on some things but not others?

So question is, is this a result of a bad stage4 build/restore or just that restore will not work on new system?

cheers
_________________
Work Station - 64bit
Gigabyte GA X48-DQ6 Core2duo E8400
8GB GSkill DDR2-1066
SATA Areca 1210 Raid
BFG OC2 8800 GTS 640mb
--------------------------------
Notebook
Samsung Q45 7100 4gb
Back to top
View user's profile Send private message
carlosman
n00b
n00b


Joined: 16 Oct 2004
Posts: 18
Location: Panama

PostPosted: Tue Dec 20, 2005 2:54 pm    Post subject: -mcpu vs. -mtune Reply with quote

First of all, Blinkeye, THANK YOU!!! Great script...

My question:

In your "GNU/Linux System Backup Script (stage4)" wiki you state:
Quote:
"march" breaks compatibility, "mcpu" doesn't. There is a simple workaround: replace "march" with "mcpu" and recompile your whole system:


I upgraded from GCC 3.3.6 to 3.4.4, as part of a general update. I have been using the -mcpu flag, following your advice. Now the GCC 3.4.4 prints this warning message:
Quote:
`-mcpu=' is deprecated. Use `-mtune=' or '-march=' instead.


I have been reading about it and it seems like the -mcpu functionality is not replicated by -mtune.

Where does that leave us for portability?

P.S.: Just after posting this, I found:
http://www.dis.com/gnu/gcc/i386-and-x86-64-Options.html
It seems like -mtune is indeed the new way of saying "-mcpu": :oops:
Quote:
-mcpu=cpu-type
A deprecated synonym for -mtune.
Back to top
View user's profile Send private message
BlinkEye
Veteran
Veteran


Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums

PostPosted: Tue Dec 20, 2005 3:30 pm    Post subject: Reply with quote

thanks, i'll update the instructions.

http://gcc.gnu.org/onlinedocs/gcc-3.4.4/gcc/i386-and-x86_002d64-Options.html wrote:
-march=cpu-type
Generate instructions for the machine type cpu-type. The choices for cpu-type are the same as for -mtune. Moreover, specifying -march=cpu-type implies -mtune=cpu-type.
-mcpu=cpu-type
A deprecated synonym for -mtune.


in short: use -mtune instead of -mcpu.
_________________
Easily backup up your system? klick
Get rid of SSH Brute Force Attempts / Script Kiddies klick
Back to top
View user's profile Send private message
billman
Tux's lil' helper
Tux's lil' helper


Joined: 21 Oct 2005
Posts: 98

PostPosted: Tue Dec 20, 2005 3:32 pm    Post subject: stage4 - newbie Reply with quote

Awsome - it works great. Me being a newbie, I was hoping for some direction. Questions:

1. How can I be certain the backup has all the required files for a complete restore - What are the necessary files it should have including "/home".

2. How can I use the application vixie-cron to do incremential backup or full back ups to a vfat drive and have it delete the past weeks or something like that?

3. If I need to restore the box and have the ability to boot from an old kernel - what would be the steps.

I know these might be silly questions but new to linux I am and determind to understand and live in the Gentoo realm I am too. Hence I have had to restore the box many times before through v2I (Windoze). I no longer want to depend on that and I want a fail safe solution.

Thanks,
Bill
Back to top
View user's profile Send private message
twitch
n00b
n00b


Joined: 10 Aug 2005
Posts: 10

PostPosted: Tue Dec 20, 2005 8:12 pm    Post subject: Hardened Gentoo Restore Reply with quote

I have had problems restoring hardened gentoo (SELinux + GRSecurity). I think I have worked through most of the kinks, here is my restore routine:

SELinux + GRSecurity Restore from stage4:
Code:
 1. boot off a live-cd and repartition your harddisks and create filesystems as necessary (remove any files left on disk)
 2. eventually reboot, using option: gentoo docache
   // If this doesn't work, ie. not enough room, you can leave the cd in and restore from a usb drive
 3. # umount /mnt/cdrom
 4. remove the live-cd and insert the cd with the stage4
 5. # mount /dev/cdrom /mnt/cdrom
 6. # mount /dev/hdaX /mnt/gentoo
 7. # mkdir /mnt/gentoo/boot
 8. # mount /dev/hdaX /mnt/gentoo/boot
 9. # tar xzvpf /mnt/cdrom/host-stage4-18.04.2005-custom.tar.gz -C /mnt/gentoo/
     or
   # tar xjvpf /mnt/cdrom/host-stage4-18.04.2005-custom.tar.bz2 -C /mnt/gentoo/
10. # mount -t proc none /mnt/gentoo/proc
11. # mount -o bind /dev /mnt/gentoo/dev
12. # mount -t selinuxfs none /mnt/gentoo/selinux
13. fix the system date using the date command
14. # chroot /mnt/gentoo /bin/bash
15. # env-update
16. # source /etc/profile
17. if in need adjust necessary files (/etc/fstab, /boot/grub/grub.conf) and/or install grub
18. # emerge sync (rebuild portage tree)
19. install grub
20. # cp /proc/mounts /etc/mtab
21. run grub-install
22. # cd /etc/security/selinux/src/policy
23. # make load
24. # make chroot_relabel
25. # exit
26. # cd /
27. # cp -r /dev/* /mnt/gentoo/dev/   
      /* Had to do this because on reboot hda could not be found, was getting a superblock error on the / partition */
28. # umount /mnt/cdrom
29. # remove backup cd
30. # umount /mnt/gentoo/boot
31. # umount /mnt/gentoo/dev
32. # umount /mnt/gentoo/proc
33. # umount /mnt/gentoo/selinux
34. # umount /mnt/gentoo
35. # reboot

When I rebooted I had to make some files suid, apparently that permission did not copy over. (I figured this out because su was not working)...  My list of files I had to edit:
   /bin/passwd
   /bin/su
   /usr/bin/crontab
   /usr/bin/gpasswd
   /usr/lib/misc/ssh-keysign
   /usr/libexec/lockspool
   /usr/sbin/unix_chkpwd

I also needed to run the following after rebooting:
   # python-updater
   # emerge  --update --deep world
   # emerge qmail (it was complaining about a log directory not existing)
   # cd /etc/security/selinux/src/policy
   # make clean
   # make install
   # make load
   # make relabel


Thanks for the script!
Back to top
View user's profile Send private message
BlinkEye
Veteran
Veteran


Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums

PostPosted: Wed Dec 21, 2005 4:49 pm    Post subject: Re: stage4 - newbie Reply with quote

billman wrote:
Awsome - it works great. Me being a newbie, I was hoping for some direction. Questions:

1. How can I be certain the backup has all the required files for a complete restore - What are the necessary files it should have including "/home".

this is what the script is for - you should not worry about what you want to backup but what you don't want. if you haven't changed anything and chose a custom stage4 where you included /home/username you should be set.

billman wrote:
2. How can I use the application vixie-cron to do incremential backup or full back ups to a vfat drive and have it delete the past weeks or something like that?

the script is for full backups only - about the syntax of a cron job you either search the web/forum or wait until i put it in my wiki. unfortunately i'm delayed as i screwed up my main workstation (and as a matter of fact i do not do backups on my own boxes ( i mean, i know what i do, right? :evil: ) - well, isn't this ironically? i'm just about to lose all my data because i can't get my raid0 up and running again.

billman wrote:
3. If I need to restore the box and have the ability to boot from an old kernel - what would be the steps.

i don't understand the question. if you have to restore your box, you have lost/screwed your data/partition whatsoever. after extracting your stage4 you'll get your kernel back and may boot from them. of course, you may boot your system with another kernel. follow the restore section - nothing changes about that. you just have to adjust your boot loader that it loads the particular kernel.

billman wrote:
I know these might be silly questions but new to linux I am and determind to understand and live in the Gentoo realm I am too. Hence I have had to restore the box many times before through v2I (Windoze). I no longer want to depend on that and I want a fail safe solution.

if you want to be sure you'll have to try it on a seperate harddisk or a partition big enough to take your extracted stage4.
_________________
Easily backup up your system? klick
Get rid of SSH Brute Force Attempts / Script Kiddies klick
Back to top
View user's profile Send private message
BlinkEye
Veteran
Veteran


Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums

PostPosted: Wed Dec 21, 2005 5:10 pm    Post subject: Re: Hardened Gentoo Restore Reply with quote

twitch wrote:
I have had problems restoring hardened gentoo (SELinux + GRSecurity). I think I have worked through most of the kinks

sorry to hear that - and i have only an answer for one problem: it's possible that some daemons do not start because you did not backup the /var/log/daemon_log directory. but you could add that directory to the $default_include_files for the next time. (on second thought it might be a good idea to do that automatically. i'll have a look into it). why you had to adjust permissions i do not understand. could it be that you extracted the stage4 without the -p option (meaning without preserving the permissions)? please extract the stage4 again (don't forget the -p/--same-permissions option) within some directory and check your permissions of /etc/passwd and so on ...
_________________
Easily backup up your system? klick
Get rid of SSH Brute Force Attempts / Script Kiddies klick
Back to top
View user's profile Send private message
Lonnie
n00b
n00b


Joined: 13 Nov 2005
Posts: 3
Location: Minnesota

PostPosted: Sun Dec 25, 2005 5:40 am    Post subject: A thank you to BlinkEye Reply with quote

Just wanted to say that I finally took the plunge and did the backup/restore. I don't know just what I was doing wrong, but the whole thing gave me fits for a week. I even had serious problems with trying to do a completely new install.

Managed to get both to reboot, but the restore was messed up on my partitions, finally figured that out today.

The new install would boot up, but the screen was impossible to read. Haven't a clue what I did wrong there.

So, today, I was going to give it one last shot before trying something else. I finally mounted ALL of my partitions after I had installed the stage4 dvd. Followed the instructions and got the restore to work. umounted ALL the partitions and rebooted.

Started startxfce4 and it acted like it was just hung, so did an 'emerge --deep --update world', the restore was already a week old.

Anyway, I had already learned that this system can be very slow coming back up from something major, so I restarted startxfce4 and when off on another project. When I came back, X was up, and as of right now, everthing I have tried is working very well, thank you.

My only Real problem today was that firefox didn't want to compile(x86_64 system). It failed on me a couple times. Found out about the 'emerge --skipfirst --resume' and got through the update world. Then I tried to emerge firefox again, and it went.

So, other than the rank-newbie errors, the stage4 work's GREAT. I went from a 100% full /usr on a 40gig HD to plenty of room on an 80gig with a 40gig just for storage. And, I was very pleasantly surprised to find that it backed up and restored Everything I was concerned about loosing.

Thanks BlinkEye, another very happy user of your great script.
_________________
No windows on my computers
Do make a living working on them though
Mostly Debian, so far
Are we having Fun Yet?
Back to top
View user's profile Send private message
mattjgalloway
l33t
l33t


Joined: 16 Mar 2004
Posts: 761
Location: Coventry, UK

PostPosted: Tue Jan 17, 2006 1:08 pm    Post subject: Reply with quote

Hey,

I just started using this script, and I have the following set:
Code:
default_exclude_list="
/dev
/lost+found
/mnt
/proc
/sys
/tmp
/var/log
/var/tmp
/var/db
/var/cache/edb
$stage4Location
`echo $CCACHE_DIR`"

default_include_files="
/dev/null
/dev/console
/home
/storage
/backup
/steam
/opt
/mnt
`find /mnt -name .keep`
/proc
/sys
/tmp
/usr/portage
/usr/src
/var/log/emerge.log
/usr/src/linux-`uname -r`/.config"


And I find that /usr/portage and all files under it are backed up!? Surely they shouldn't be, only the directory structure...
_________________
AMD64 3200+, 1024MB RAM, Gentoo Linux
MacBook Core Duo, 1024MB RAM, Leopard
Back to top
View user's profile Send private message
BlinkEye
Veteran
Veteran


Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums

PostPosted: Tue Jan 17, 2006 2:17 pm    Post subject: Reply with quote

you removed /usr/portage (which you should readd) from default_exclude_list:
mkstage4.sh wrote:

# these files/directories are always excluded. don't add trailing slashes.
# don't touch it unless you know what you are doing!
# /var/db and /var/cache/edb are intentionally added here. they are listed
# in $default_include_folders
default_exclude_list="
/dev
/lost+found
/mnt
/proc
/sys
/tmp
/usr/portage
/usr/src
/var/log
/var/tmp
/var/db
/var/cache/edb
$stage4Location
`echo $CCACHE_DIR`"

_________________
Easily backup up your system? klick
Get rid of SSH Brute Force Attempts / Script Kiddies klick
Back to top
View user's profile Send private message
mattjgalloway
l33t
l33t


Joined: 16 Mar 2004
Posts: 761
Location: Coventry, UK

PostPosted: Tue Jan 17, 2006 5:33 pm    Post subject: Reply with quote

Hi.

Thanks for that! I really have no idea why I'd deleted them from there, but that makes sense now!

Thanks!
_________________
AMD64 3200+, 1024MB RAM, Gentoo Linux
MacBook Core Duo, 1024MB RAM, Leopard
Back to top
View user's profile Send private message
mattjgalloway
l33t
l33t


Joined: 16 Mar 2004
Posts: 761
Location: Coventry, UK

PostPosted: Tue Jan 17, 2006 6:17 pm    Post subject: Reply with quote

That seems to be working now, but I thought of another thing...

Perhaps there could be a progress bar incorporated into the script... It could just be along the lines of files left to do / total files to get the percentage done. Obviously a better way would be the size of files, but it'd be a good start at least.

Is that do-able do you think?
_________________
AMD64 3200+, 1024MB RAM, Gentoo Linux
MacBook Core Duo, 1024MB RAM, Leopard
Back to top
View user's profile Send private message
carpman
Advocate
Advocate


Joined: 20 Jun 2002
Posts: 2202
Location: London - UK

PostPosted: Tue Jan 17, 2006 6:42 pm    Post subject: Reply with quote

hello, ok just done a stage4 backup and restore so i could change FS type and layout.

It worked ok except for couple of issues, one was firefox profile lock file which needed deleting as i could not load firefox as it complained that profile was in use. My touchpad stopped working but just needed to change device type from event one to auto.

I do have one more issue and that is cpufreqd, i have it set so when computer at idel it drops to lowest cpu setting 533mhz and when under load it goes up in stages to 1600mhz, thing is that after restore it has not gone higer than 533 even though i now compiling perl update?

Not sure what is occuring but was working fine before restore, have checked cpifreqd conf.

cheers
_________________
Work Station - 64bit
Gigabyte GA X48-DQ6 Core2duo E8400
8GB GSkill DDR2-1066
SATA Areca 1210 Raid
BFG OC2 8800 GTS 640mb
--------------------------------
Notebook
Samsung Q45 7100 4gb
Back to top
View user's profile Send private message
BlinkEye
Veteran
Veteran


Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums

PostPosted: Tue Jan 17, 2006 6:55 pm    Post subject: Reply with quote

carpman wrote:
hello, ok just done a stage4 backup and restore so i could change FS type and layout.

It worked ok except for couple of issues, one was firefox profile lock file which needed deleting as i could not load firefox as it complained that profile was in use. My touchpad stopped working but just needed to change device type from event one to auto.

I do have one more issue and that is cpufreqd, i have it set so when computer at idel it drops to lowest cpu setting 533mhz and when under load it goes up in stages to 1600mhz, thing is that after restore it has not gone higer than 533 even though i now compiling perl update?

Not sure what is occuring but was working fine before restore, have checked cpifreqd conf.

cheers

i guess you tried to re-emerge cpufreqd? maybe a device is missing and hence cpufreqd cannot reset the speed. the /dev folder does not be backed up and as of udev there may be some nodes which have to be recreated?
_________________
Easily backup up your system? klick
Get rid of SSH Brute Force Attempts / Script Kiddies klick
Back to top
View user's profile Send private message
carpman
Advocate
Advocate


Joined: 20 Jun 2002
Posts: 2202
Location: London - UK

PostPosted: Tue Jan 17, 2006 7:29 pm    Post subject: Reply with quote

Seems to have started working but still have one issue, /var/log/portage i created this and given it lots chmod options, it is currently 777 but still emerges complain it is not writable? i set it to owner/group 'portage' as is on other systems but no joy?

PS great script otherwise :)

Cheers
_________________
Work Station - 64bit
Gigabyte GA X48-DQ6 Core2duo E8400
8GB GSkill DDR2-1066
SATA Areca 1210 Raid
BFG OC2 8800 GTS 640mb
--------------------------------
Notebook
Samsung Q45 7100 4gb
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page Previous  1, 2, 3 ... 13, 14, 15, 16, 17, 18  Next
Page 14 of 18

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum