Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
HOWTO: Create an auto-restore LiveCD for use with rsnapshot
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
ryouga42
n00b
n00b


Joined: 21 Oct 2004
Posts: 19

PostPosted: Fri Nov 05, 2004 6:43 am    Post subject: HOWTO: Create an auto-restore LiveCD for use with rsnapshot Reply with quote

A while back I started looking for a way to ensure that the server I've spent so long building here would stay alive longer than about a month after I leave. One of the things I did was set up daily rsnapshot backups of essentially the entire root FS and keep last 7 days, last 4 weeks, last 3 months. But this isn't really enough - I'd then have to write a guide on exactly how to fix the system if it breaks, what to use, etc.

I decided a bootable CD that would determine what backups could be restored and do it all automagically would be the best way to handle the problem. It was also a very interesting exercise in hacking the LiveCD.

Essentially I decided I'd use the Gentoo LiveCD because it does a very good job of booting and getting everything configured "out of the box". All I had to do was find a way to stick my script at the end of the boot sequence. /etc/profile does the trick. Anyway, on with the HOWTO.

I did the script in Perl, so we'll need to do some extra hacking to get Perl to actually work on the LiveCD.

Please remember that this is a script I wrote specifically for my server's backup routine. While it may work on your system, probably not without some tweaks - any genericness in the script is there because it was convenient for me at the time. (I created this on a live server, so I needed to test it by restoring to secondary drives, etc) You will most likely need to hack it a little to get it to work.

First, you need to get a LiveCD or image. Preferably an image, but as long as you know how to extract the root filesystem and then re-compress it later you'll be fine. Otherwise, follow me with the 2004.2 LiveCD:

Get a copy of the "minimal" LiveCD. Also download a stage* tarball for your system. It doesn't matter what stage it is.

You'll also need to emerge squashfs-tools. By the same token you need squashfs support either in your kernel or as a module. And make sure you have loop support.

Mount your CD/image somewhere (/mnt/cdrom works well):

Code:
mount /path/to/cdimage /mnt/cdrom -o loop


And now mount the livecd.squashfs file:

Code:
mount /mnt/cdrom/livecd.squashfs /mnt/squashfs -o loop


Now we need to copy the root filesystem somewhere and work on it.

Code:
cp -a /mnt/squashfs ~


Okay, now comes a little bit of trickiness. First, you'll need Perl in the root FS. I find that just copying all of perl's executables and its own modules seems to work, i.e. the actual shared libraries are there. We'll test this in a minute, however. First, copy Perl and grub to the squashfs:

Code:
cp -a /usr/bin/perl* ~/squashfs/usr/bin
cp -a /usr/lib/perl* ~/squashfs/usr/lib
cp -a /sbin/grub* ~/squashfs/sbin


Great. This might copy some extra files that we don't need (in fact it does) but it doesn't matter unless you're looking to get this thing on a mini-CD or something. If so, I suggest you start pulling Perl modules out - I only used File::Path in the script so you can probably remove pretty much all of them except that one. I suggest testing it afterwards, however.

Easy way to test that all your libraries are in place:
Code:
chroot ~/squashfs /bin/bash
perl
grub


If you didn't get error messages that means all your libraries are okay. If not, do the following to see what libraries you need:
Code:
exit (if you haven't left the chroot yet)
ldd /usr/bin/perl
ldd /usr/bin/grub


And copy those files to ~/squashfs/lib. I didn't have a problem so I assume all the libraries are already in place on the default liveCD.

Now, there are a couple of housekeeping issues to take care of. First of all, the script doesn't care what tarball you have. So, copy that tarball to ~/squashfs/root/basesystem.tar.bz2 - you can have anything here, if you want to make it, say, a mini-CD, perhaps you only need the directories you don't back up regularly.

Second, you need to save a partition table for the disk you will be restoring to. You'll save this in the root of the disk you're using to back up, so mount it somewhere, then:
Code:
sfdisk -d /your/system/disk > /your/backup/disk/.partitiondata


So for example:
Code:
sfdisk -d /dev/sda > /mnt/backup/.partitiondata


Please note that sfdisk does not handle non-contiguous partitions well. For instance, if you have your disk set up such that all your normal, "data" partitions are at the beginning of the disk, and your swap is at the end of the disk, if there is any free, non-partitioned space in between, weird things can happen. Just make a partition separating them when you do this, if you insist on having this setup.

Also note that the script relies on fstab to know what your original filesystems were - "auto" will not work - and it can only create ext2, ext3, reiser and swap because those are the only four I cared about. More are easily added, look into the script for details.

Now make /mnt/backup and /mnt/restore:

Code:
mkdir ~/squashfs/mnt/backup ~/squashfs/mnt/restore


Set up etc/profile to autoload the script. Do this by adding this to the end of ~/squashfs/etc/profile:
Code:
case `tty` in
   /dev/tty[1])
      echo -n "Starting the Wayback Machine..."
      /sbin/wayback.pl
   ;;
esac


This will only load the script on the first console, so you'll still be able to login on all the other ones.

Finally, copy and paste this into a file named "wayback.pl" in sbin/.
Code:
#!/usr/bin/perl -w

use strict;
use File::Path qw(mkpath);

my $backup_disk = "/dev/sdb1";
my $backup_dir = "/mnt/backup";
my $backup_name = "avomrtg";
my $restore_target = "/dev/sdc";

# Used to sort date values.
my %datevals = (
   "daily"     => 1,
   "weekly"    => 2,
   "monthly"   => 3,
   "yearly"    => 4,
);

system("mount $backup_disk $backup_dir");

# Get directory listing and sort it:
opendir(D, $backup_dir);
my @list =  sort dsort
            grep {exists $datevals{(split /\./)[0]}}
            grep {m/\.\d+$/}
            readdir(D);
closedir(D);

print "Wayback Machine completed initialization. Please choose the backup you want to restore:\n";

my $counter = 0;

# Show them the (sorted) list
foreach my $backup (@list) {
   $counter++;
   print "$counter) $backup\n";
}

# And ask for their choice
my $choice;
while (1) {
   print "Enter your selection: ";
   chomp ($choice=<STDIN>);

   last if ($choice !~ /\D/ &&
      $choice > 0 &&
      $choice <= $counter);

   print "Invalid input, please try again.\n";
}
$choice--;

# Now confirm everything:
print "I am about to restore backup: $list[$choice] to the disk $restore_target.\n";
print "\033[31;1mTHIS WILL DESTROY ALL DATA ON THE DISK $restore_target!!!!!\033[0m\n";
print "If you want to cancel, press CTRL+ALT+DEL to reboot and eject the CD.\n";
print "If you are sure you want to continue, type \"yes\" now: ";

while (1) {
   chomp (my $y=<STDIN>);
   last if ($y eq "yes");

   print "If you want to abort, press CTRL+ALT+DEL now. If you would like to continue, type \"yes\": ";
}

# Restore partition data
system("sfdisk $restore_target < $backup_dir/.partitiondata");
die "Couldn't restore partition table" if ($? != 0);

# and remake partitions
open FSTAB, "$backup_dir/$list[$choice]/etc/fstab" or die "Couldn't open old fstab: $!";

my %mountpoints;
while (<FSTAB>) {
   next unless m#^$restore_target#;
   my @line = split;

   # If you're using something other than ext2, ext3 or reiser add it in
   if ($line[2] eq "ext2") {
      system("mke2fs -q $line[0]");
      die "Couldn't make ext2 file system on $line[0]" if ($? != 0);
   } elsif ($line[2] eq "ext3") {
      system("mke2fs -qj $line[0]");
      die "Couldn't make ext3 file system on $line[0]" if ($? != 0);
   } elsif ($line[2] eq "reiserfs") {
      system("mkreiserfs -q $line[0]");
      die "Couldn't make reiser file system on $line[0]" if ($? != 0);
   } elsif ($line[2] eq "swap") {
      system("mkswap $line[0]");
      die "Couldn't make swap on $line[0]" if ($? != 0);
   } else {
      die "Unsupported file system $line[2] on $line[0]";
   }

   # Now add it to the hash
   $mountpoints{$line[1]} = $line[0] unless ($line[2] eq "swap");
}
close FSTAB;

# Get the root filesystem mounted first
system("mount $mountpoints{\"/\"} /mnt/restore");
die "Couldn't mount restore target's root filesystem" if ($? != 0);

# Now do all the other ones
delete $mountpoints{"/"};
for my $mountpoint (keys %mountpoints) {
   mkpath("/mnt/restore$mountpoint");
   -d "/mnt/restore$mountpoint" or die "Couldn't create /mnt/restore$mountpoint";
   system("mount $mountpoints{$mountpoint} /mnt/restore$mountpoint");
   die "Couldn't mount $mountpoints{$mountpoint} on /mnt/restore$mountpoint" if ($? != 0);
}

# Great, now restore the system
print "Done restoring disk partitions. Now restoring files...\n";
sleep 5;
system("tar xvjpf /root/basesystem.tar.bz2 -C /mnt/restore");
system("cp -afv $backup_dir/$list[$choice]/* /mnt/restore/");
die "Couldn't cp all files!" if ($? != 0);

# And run grub
system("grub-install --root-directory=/mnt/restore/boot $restore_target");
die "Didn't install grub on $restore_target" if ($? != 0);

print "The system should now be restored to a running state. Please press CTRL+ALT+DEL to reboot and remove the disc.\n";

sub dsort {
   my @a = split /\./, $a;
   my @b = split /\./, $b;

   return $a[1] <=> $b[1] if ($a[0] eq $b[0]);
   return $datevals{$a[0]} <=> $datevals{$b[0]};
}


Change it to fit your installation. Of note is the date hash table which allows you to add and remove different "intervals" - I don't back up any more than daily so I didn't put it in there, but this table also affects what directories the script sees. Changing the top variables and this should get you most of the way to what you need. My setup isn't complicated, but you need to make sure that if your daily backups are in, say, daily.0/, that there isn't a hostname or something weird stuck in there that'll cause the script to fail. Also it just copies the contents straight to the root of the restored disk, so make sure you back specific files up to wherever they need to go in the tree.

Your squashfs is complete - you can change /etc/motd to whatever you like and also hack the initrd to make a different boot logo, but that's outside the scope of this guide (and there are many better ones out there). Now re-create the squashfs:

Code:
mksquashfs ~/squashfs ~/livecd.squashfs


This will take quite a long time, but it does do a good job. Now recreate the LiveCD:

Code:
cp -a /mnt/cdrom ~
cd
mv livecd.squashfs cdrom
mkisofs -o livecd.iso -r -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table cdrom


And burn "livecd.iso" using your burning program of choice.
Back to top
View user's profile Send private message
mach.82
Tux's lil' helper
Tux's lil' helper


Joined: 30 Oct 2003
Posts: 75
Location: 43°N/79°W

PostPosted: Thu Nov 11, 2004 1:01 am    Post subject: HOWTO: Create an auto-restore LiveCD for use with rsnapshot Reply with quote

Very nice HowTo! :D
Thanks a 1,000,000!
Back to top
View user's profile Send private message
deque
n00b
n00b


Joined: 28 Apr 2003
Posts: 53

PostPosted: Wed Dec 08, 2004 5:48 pm    Post subject: Reply with quote

Iam working on a project quite similar to this one. Whenever I try to copy things from the mounted squashfs filesystem, I get a segmentation fault. Have any of your run into this?

EDIT: I've noticed that it seems to happen whenever it copies any of the .keep files that in there...
_________________
-Regards-

-Deque-
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum