Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
A Gentoo auto-installer? Impossible!? Guess not.
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
roothorick
Tux's lil' helper
Tux's lil' helper


Joined: 30 May 2004
Posts: 83
Location: Menasha, WI

PostPosted: Tue Jul 27, 2004 7:47 pm    Post subject: A Gentoo auto-installer? Impossible!? Guess not. Reply with quote

DISCLAIMER
THIS AUTOMATIC INSTALLATION SCRIPT WAS DESIGNED SPECIFICALLY FOR THE GHETTOCLUSTER PROJECT AND WILL PROBABLY NOT FUNCTION PROPERLY UNMODIFIED IN OTHER SITUATIONS. USE AT YOUR OWN RISK!


Considering the volume of donations I'm expecting for the GhettoCluster project, it's more than reasonable to think that I need some way of installing Gentoo automatically.

For those that don't know what GhettoCluster is, it's a project being run exclusively by me with two goals:
  1. To prove that you can build a cluster dirt cheap, if not absolutely free, by soliciting donations of computers nobody wants.
  2. To provide the necessary knowledge to do just that through documentation as well as providing whatever tools necessary that aren't already available.

A more refined version of this script will be released as part of a package later under the second goal. (This also explains why it's MIT licensed and not public domain or "lazy-license".)

Anyway, after about 18 tests, I finally have my script working. Note again that it probably won't work on your setup. It's meant merely as an example to those wishing to uninteractively install Gentoo on a computer.

Here's the script. Be ready, it's over 250 lines, and it's a VERY early version:
Code:
#!/bin/sh
# Batched installation script for installing Gentoo Linux semi-autonomously.
# by Ben Anderson
# This script has been specifically written for use in the Ghetto Cluster
# project or a similar project. Adjust to your needs if you want. In particular,
# there's a few IPs you'll probably have to change, do a search for "10." ;)

# License: (for the lazy, it's the MIT license.)
# Copyright (c) 2004 Ben Anderson and the GhettoCluster project.

# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:

# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.

# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.

STAGEFROM=1 # Stage we're starting at.
ARCH="i686" # This is modified on a per-node basis.
CPU="pentium2" # Also modified on a per-node basis.
OURIP="10.0.1.1" # What our IP will be when we're added to the network.
# Nodes aren't given hostnames and are only known by IP. There's too many of
# them to name all of them!

DATADIR="/root/ainstall" # Where did I stash all the other necessary files?

CLUSTERMASTER="10.0.0.1" # The IP of your "cluster master".

USE="-* pam ssl readline ncurses crypt"

NUMPROCS=3 # Number of processors, plus the computer that's getting Gentooized.

LOCALPROCS=1 # Number of processors in the box that's getting Gentooized.

INSTALLDRV="/dev/hda"

KERNELSPEC="2.6.7-pentium2-piix"

part1()
{
    # Apply the partition table, create the filesystems, and mount 'em!
    # Note that the table is read from a file in $DATADIR that should already
    # exist.
    # We set up the hard drive and get swap going IMMEDIATELY. We might need it!
    echo "Writing partition table..."
    sfdisk $INSTALLDRV < $DATADIR/parttable
    echo "Creating swap signature:"
    mkswap $INSTALLDRV"2"
    echo "Enabling swap..."
    swapon $INSTALLDRV"2"
    echo "Creating filesystems:"
    echo "/:"
    mke2fs -j $INSTALLDRV"3"
    echo "/boot:"
    mke2fs $INSTALLDRV"1"
    echo -n "Mounting filesystems in the new environment: "
    mount $INSTALLDRV"3" /mnt/gentoo
    echo -n "/ "
    mkdir /mnt/gentoo/boot /mnt/gentoo/proc
    mount $INSTALLDRV"1" /mnt/gentoo/boot
    echo -n "/boot "
    mount -t proc none /mnt/gentoo/proc
    echo -n "/proc "
    echo

    # Note that it pulls the stage tarball off the cluster master instead of an
    # Gentoo mirror. This is intentional because it eases the load on Gentoo
    # mirrors.

    # TODO:
    # $ARCH will eventually be used to partially create a make.conf on the fly.
    if [ $STAGEFROM = "1" ]; then
   STARCH="x86"
    else
        STARCH=$ARCH
    fi
    echo "Grabbing stage from master..."
    wget ftp://ghettocluster@$CLUSTERMASTER/stages/stage$STAGEFROM-$STARCH.tar.bz2 \
        -O /mnt/gentoo/stage.tar.bz2
    echo "Unpacking stage..."
    cd /mnt/gentoo && tar xvjpf /mnt/gentoo/stage.tar.bz2
    # Discard the stage now that it's unpacked.
    rm /mnt/gentoo/stage.tar.bz2

    # copy resolv.conf. It's really obvious this is based on the handbook,
    # isn't it?
    cp -L /etc/resolv.conf /mnt/gentoo/etc/resolv.conf

    # I make a kernel beforehand for the installation on non-master nodes.
    # Here we grab that kernel and go.
    # TODO:
    # I should have a hardware-universal kernel for every subarch instead of
    # compiling on a per-node basis. I'm guessing it'll be PII-oriented
    # guessing by the nature of most of the donations.
    wget \
        ftp://ghettocluster@$CLUSTERMASTER/kernels/kernel-$KERNELSPEC.tar.bz2 \
        -O $DATADIR/kernel.tar.bz2
    cd /mnt/gentoo && tar jxvf $DATADIR/kernel.tar.bz2

    # Generate the fstab. This is fun!
    echo $INSTALLDRV"1 /boot ext2 noauto,noatime 1 2" >  /mnt/gentoo/etc/fstab
    echo $INSTALLDRV"3 /     ext3 noatime        0 1" >> /mnt/gentoo/etc/fstab
    echo $INSTALLDRV"2 none  swap sw             0 0" >> /mnt/gentoo/etc/fstab

    # Copy ourselves into the chroot jail and pick up from there
    cp $DATADIR/installscript.sh /mnt/gentoo/installscript.sh
    chroot /mnt/gentoo /bin/bash /installscript.sh part2
}
part2()
{
    # My hunch is this should go before the env update. Correct me if I'm wrong.
    # Just so the nodes know who the cluster master is, we tell them. This is
    # needed early for emerge sync.
    echo "10.0.0.1 memnarch.gc memnarch" >> /etc/hosts
    # Not sure if it's important inside a shell script like this, but it's good
    # to have anyway.
    env-update
    source /etc/profile

    # Here we go! Here we go! Custom make.conf, coming right up! (part 1)
    # First of all, if it's a stage2 or stage3, we can't change the CHOST, so
    # check for that.
    if [ $STAGEFROM -gt 1 ]; then
       CHOST=`grep CHOST | cut -f2 -d = | cut -f2 -d \"`
    else
       CPUFAMILY=`cat /proc/cpuinfo | grep 'cpu family' | cut -f 2 -d : | cut -b 2`
       CHOST="$ARCH-pc-linux-gnu"
    fi

    # Now figure out the best optimization for the CPU we're on
    # FIXME I know there's more cases than what's supported here...
    CPUBRAND=`cat /proc/cpuinfo | grep vendor_id | cut -f 2 -d : | cut -b 2-`
    # FIXME This only works as long as the kernel tells cache size in KB.
    CPUCACHE=`cat /proc/cpuinfo | grep 'cache size' | cut -f 2 -d : \
        | cut -b 2- | cut -f 1 -d " "`
    if [  $CPUBRAND = "GenuineIntel" -a $CPUCACHE -lt 512 ]; then
        OPTIM="s"
    else
        OPTIM="2"
    fi

    # Now we FINALLY actually write it!
    echo "CFLAGS=\"-O$OPTIM -march=$CPU -fomit-frame-pointer -pipe -ftracer"\
        "-frename-registers\"" > /etc/make.conf
    echo "CHOST=\"$CHOST\"" >> /etc/make.conf
    echo "USE=\"$USE\"" >> /etc/make.conf
    echo "CXXFLAGS=\"\${CXXFLAGS}\"" >> /etc/make.conf
    echo "SYNC=rsync://10.0.0.1/gentoo-portage" >> /etc/make.conf

    # New make.conf, let's reload the environment.
    source /etc/profile

    # Note that this doesn't pull off the regular portage mirror, but instead
    # our internal private mirror, that is updated once a week.
    # TODO:
    # I'm hoping that eventually only the cluster master will have the portage
    # tree, and the rest of the computers just install the packages that are
    # handed to them. We'll get there.
    emerge sync

    # Pull ourselves into the loop. First grab distcc and then register
    # ourselves as a node.
    if [ $STAGEFROM -eq 1 ]; then
        USE="-*" MAKEOPTS=$(($LOCALPROCS*2+1)) emerge --nodeps distcc
    else
        MAKEOPTS=$(($LOCALPROCS*2+1)) emerge distcc
    fi

    # For the curious, in the below line you can see two VERY little-known
    # features in use -- one, that you can do whatever is legal in bash in
    # make.conf, and two, bash $(()) arithmetic evaluation. Together they are
    # somewhat handy when your cluster scales up and down rapidly.
    echo "MAKEOPTS=\"-j$(($NUMPROCS*2+1))\"" >> /etc/make.conf
    echo "FEATURES=\"distcc\"" >> /etc/make.conf
    if [ $STAGEFROM -eq 1 ]; then
        echo "DISTCC_DIR=\"${PORTAGE_TMPDIR}/.distcc\"" >> /etc/make.conf
   echo "distcc:x:240:2:distccd:/dev/null:/bin/false" >> /etc/passwd
    fi
   
    # New make.conf, let's reload the environment.
    source /etc/profile
   
    # STAGE ONE TO STAGE TWO: bootstrap!
    cd /usr/portage && scripts/bootstrap.sh
   
    # STAGE TWO TO STAGE THREE: emerge system!
    emerge system

    # Network configuration is so basic, I'm putting it inside the shell script
    # :)
    echo "iface_eth0=\"$OURIP broadcast 10.0.0.255 netmask 255.0.0.0\"" > /etc/conf.d/net
    echo "gateway=\"eth0/10.0.0.1\"" >> /etc/conf.d/net
    rc-update add net.eth0 default
    echo $OURHOST > /etc/hostname
    echo "ghettocluster" > /etc/dnsdomainname

    # Install various necessities.
    emerge grub syslog-ng vixie-cron
    rc-update add syslog-ng default
    rc-update add vixie-cron default
   
    # While we're at it, if we didn't earlier, recompile distcc.
    if [ $STAGEFROM -eq 1 ]; then
        emerge distcc
    fi
   
    rc-update add distccd default

    # Bootloader installation.
    cat << EOF | grub --batch
root (hd0,0)
boot (hd0)
quit
EOF

    # Generate grub.conf
    KERNELVERSION=`echo $KERNELSPEC | cut -f 1 -d -`
    cat << EOF > /boot/grub/grub.conf
timeout 5
default 0
# There is no fallback.
title Gentoo Linux $KERNELVERSION
root (hd0,0)
kernel /kernel-$KERNELVERSION root=/dev/hda3
EOF

    # Generate the fstab. This is fun!
    echo $INSTALLDRV"1 /boot ext2 noauto,noatime 1 2" >  /etc/fstab
    echo $INSTALLDRV"3 /     ext3 noatime        0 1" >> /etc/fstab
    echo $INSTALLDRV"2 none  swap sw             0 0" >> /etc/fstab

    # Notes to self because this script isn't complete
    echo "Don't forget to set the root password and make a regular user"
    echo "Also don't forget to add the ethernet module to autoload"
    echo "Especially don't forget to add packages and configure this system"\
        "for use as a node"
}

if [ $1 = "part2" ]; then
    part2
else
    part1
fi

_________________
Note: This user has been arrested under the DMCA for copyright infringement based on a complaint from The Inernational Cliche Company. He is also facing charges for violating US patents describing the encoding of text in digital form.
Back to top
View user's profile Send private message
_dook_master_
Apprentice
Apprentice


Joined: 16 Sep 2003
Posts: 295
Location: Isla Vista, CA

PostPosted: Tue Jul 27, 2004 9:19 pm    Post subject: Reply with quote

That is hot! :P
Back to top
View user's profile Send private message
allucid
Veteran
Veteran


Joined: 02 Nov 2002
Posts: 1314
Location: atlanta

PostPosted: Tue Jul 27, 2004 11:32 pm    Post subject: Reply with quote

this and this might be of use to you (two other examples).
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum