Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Script: Update Servers when adding additional Distcc servers
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
mozingod
n00b
n00b


Joined: 15 Jun 2003
Posts: 64
Location: Ohio

PostPosted: Thu Dec 11, 2003 2:37 pm    Post subject: Script: Update Servers when adding additional Distcc servers Reply with quote

I started making a Distcc "cluster" at work to help support our 7+ production servers on their weekend updates. After I made the first few servers I realised that everytime I added another Distcc server I'd have to go around and update both the make.conf and distcc hosts command on every server (both production and distcc servers). Annoying? You bet'cha. So I thought I'd make a little script to help me along with it and share it with the Gentoo community :)

You'll need to setup passwordless SSH for this to work best. For this example I'll say the server you're running this from is Distcc1, and targeting Bind1.

First you'd generate a key on Distcc1:
Code:
ssh-keygen -t rsa

Hit enter for all questions to generate the default file name along with no passphrase. After that, run the following command (you'll need to enter yes to add Bind1 to the list of known servers, and enter its root password twice):
Code:
scp ~/.ssh/id_rsa.pub bind1:~/.ssh/distcc1 ; ssh bind1 "cat ~/.ssh/distcc1 >> ~/.ssh/aurthorized_keys2 ; rm ~/.ssh/distcc1"

Do that from the first distcc server for all the servers in your network. This will allow you to scp and ssh into them without a password.

Finally, here's the script:
Code:
#!/bin/bash
# Runs through all computers listed in the distccClients file,
#  updating them to take advantage of any newly added distcc
#  servers to the network

########## CHANGE THESE BEFORE RUNNING ##########
HostList="192.168.20.1 192.168.20.2"   # List of distcc servers
NewCPUCount="6"      # New CPU count (number of distcc servers + 1)
#################################################

PID=$$
LogFile=$PWD/distccUpdate.log
ServerList="distccClients"
Error=0

if [ ! -e $LogFile ]
then
   touch $LogFile
fi

echo "Started logging on `date`" >> $LogFile

echo "Script starting..."
echo ""

mkdir /tmp/$PID

getFiles()
{
   if scp $server:/etc/make.conf make.conf &> /dev/null
   then
      echo "    Make.conf copied to local machine OK" >> $LogFile
   else
      echo "!!! Make.conf copy to local machine FAILED with error code $?" >> $LogFile
      Error=1
   fi
}

updateFiles()
{
   if echo $HostList > hosts
   then
      echo "    Distcc hosts file created OK" >> $LogFile
   else
      echo "!!! Distcc hosts file creation FAILED with error code $?" >> $LogFile
      Error=1
   fi

   if sed "s/-j[1234567890]/-j$NewCPUCount/g" -i make.conf &> /dev/null
   then
      echo "    Make.conf updated OK" >> $LogFile
   else
      echo "!!! Make.conf update (sed) FAILED with error code $?" >> $LogFile
      Error=1
   fi
}

replaceFiles()
{
   if scp make.conf $server:/etc/make.conf &> /dev/null
   then
      echo "    Make.conf copied to remote machine OK" >> $LogFile
   else
      echo "!!! Make.conf copy to remote machine FAILED with error code $?" >> $LogFile
      Error=1
   fi

   if scp hosts $server:/etc/distcc/hosts &> /dev/null
   then
      echo "    Distcc hosts file copied to remote machine OK" >> $LogFile
   else
      echo "!!! Distcc hosts file copy to remote machine FAILED with error code $?" >> $LogFile
      Error=1
   fi
}

for server in `cat $ServerList`
do
   mkdir /tmp/$PID/$server
   cd /tmp/$PID/$server

   echo "" >> $LogFile
   echo "************ $server ************" >> $LogFile
   echo "Processing server $server..."

   getFiles
   updateFiles
   replaceFiles

   if [ $Error -eq 0 ]
   then
      echo "" >> $LogFile
      echo "Server $server finished OK." >> $LogFile
   else
      echo "" >> $LogFile
      echo "!!! Server $server update FAILED." >> $LogFile
   fi

   Error=0
done

rm -fr /tmp/$PID

if [ $? -eq 0 ]
then
   echo ""
   echo "Script finished successfully."
else
   echo ""
   echo "!!! Encountered error $?."

   exit 1
fi

echo "" >> $LogFile
echo "" >> $LogFile
echo "==========================================================================" >> $LogFile
echo "" >> $LogFile
echo "" >> $LogFile

exit 0

This reads in a file named "distccClients" in the same directory as the script that contains a list of servers in your network (one server name/IP per line). Notice this sets all the servers to the same number of processors, and along with only including the distcc server IP's in the list will allow the production servers to use only the distcc server for compiling, taking the load off themselves. The distcc servers, however, will use their own CPU when they update. Works out quite nicely if you ask me.

I'm sure there's a better way to do this, but I don't know of it. I hacked this together and its worked fine for me. If you see a way to improve it please post, I'd really like to know.

Also note you don't really need to setup the passwordless SSH, but for each server you'll have to enter the root password, which can get pretty annoying pretty quick.

Don't forget to check the generated log file (<scriptname>.log) for errors!
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum