Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
perl-cleaner and paludis, a small hack of mine
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Unsupported Software
View previous topic :: View next topic  
Author Message
Styrsven
n00b
n00b


Joined: 30 Aug 2009
Posts: 5

PostPosted: Sun Aug 30, 2009 8:17 am    Post subject: perl-cleaner and paludis, a small hack of mine Reply with quote

As a paludis user, I was missing the possibility to use the perl-cleaner script. When looking at it, I realized that it wasn't rocket science, so I hacked up my own paludis-aware version. If anyone is brave enough to test it, I would appreciate some feedback...

UPDATE: I cleaned up the code somewhat (my changes, that is), and some clarification: I have only added a paludis parameter, and implemented it in the code. I have not done any other changes to the logic of the script. The 'ask' parameter does not exist in paludis, so it is not forwarded, but apart from that is handled just as before in the script.

I have successfully upgraded my system to perl 5.10.1 from perl-experimental with this script, so I think it's ok. I have not tested it with an 'emerge' system though.

This version is downloadable from http://pastebin.com/f74264309

Code:

#!/bin/sh

# Paludis-aware version
#- a hack, second try
# Version 1.4
#- patches from Wade Fitzpatrick for equery and fgrep suggestions (bug 125674),
#- patches from Joerge Plate for grep being called with too many arguments (bug 114155)
#- finally updated emerge syntax, brought up by nyhm (with some input from cab) in bug 128130.
# Version 1.3
# sort of a changelog if you want to call it that...
# grep -E '/usr/lib*/perl5/' */*/CONTENTS > to make my list :)
# version 1.2 - swtaylor gave some good pointers on making the tmp files, as well as reminding me of grep's -f functionality :)
# version 1.1 - Mr. Bones gave a lot of good input on cleaning up the script
# Version 1 - stuff

# First and foremost - make sure we have a perl to work with...
PERL=$(which perl)
if [ "${PERL}x" = "x" ]; then
   echo "NO PERL INSTALLED!! (at least not in your path)"
   exit
fi
eval $(perl '-V:version')
PERL_VERSION=${version}
gPERL_VERSION=`echo $PERL_VERSION|sed -e 's|\.|\\\.|g'`
. /etc/init.d/functions.sh || {
        echo "$0: Could not source /init.d/functions.sh!"
        exit 1
}

TMPDIR=${TMPDIR:-/tmp}

DATESTAMP=$(date +"%Y%m%d%H%M%S")
LOG=$(mktemp ${TMPDIR}/perl-cleaner.log.$DATESTAMP.XXXXXXXXXX)
PAGER=${PAGER:-more}

# Set up our temporary files
MODULES_LIST=$(mktemp ${TMPDIR}/modules.list.XXXXXXXXXX)
EBUILDS_PREINSTALL=$(mktemp ${TMPDIR}/ebuilds.preinstall.XXXXXXXXXX)
EBUILDS_ORDERED=$(mktemp ${TMPDIR}/ebuilds.ordered.XXXXXXXXXX)
EBUILDS_REINSTALL=$(mktemp ${TMPDIR}/ebuilds.reinstall.XXXXXXXXXX)

postclean() {
   for FILE in ${MODULES_LIST} ${EBUILDS_PREINSTALL} ${EBUILDS_ORDERED} ${EBUILDS_REINSTALL}; do

      if [ -f $FILE ]; then
         rm -f $FILE
      fi

   done

   if [ -s $LOG ]; then
      echo
      echo "For a complete log, please read $LOG"
      echo
   else
      if [ -f $LOG ]; then
        rm -f $LOG
      fi
   fi
}

# This is to clean out the old .ph files generated in our last perl install
ph_clean() {
   echo ""
   echo "$(date) : Beginning a clean up of .ph files" | tee -a $LOG
   echo "Excluding perl-${PERL_VERSION} from cleaning" | tee -a $LOG

   local PV=${version}
   local gPV=`echo $PV|sed -e 's|\.|\\\.|g'`
   INC=$(perl -e 'for $line (@INC) { next if $line eq "."; next if $line =~
   m/'${PV}'/; print "$line\n" }')
   if [ "${INC}x" != "x" ]; then
   echo "Locating ph files for removal"
      for file in $(find $INC -name "*.ph" -type f 2>/dev/null); do
         if [ ! $(echo "$file"|grep $gPV) ]; then
            echo ""
            echo "$(date) : Removing old ph file: $file" | tee -a $LOG
            rm $file
         fi
      done
   fi

   # Silently remove those dirs that we just emptied
   find $INC -depth -type d 2>/dev/null | grep -v $gPV | xargs -r rmdir 2>/dev/null
}

# Generate ph files; this is useful if we've upgraded packages with headers so that perl knows the new info
ph_update() {
   echo ""
   echo "$(date) : Updating ph files" | tee -a $LOG
   cd /usr/include; h2ph * | tee -a $LOG
   cd /usr/include; h2ph -r sys/* arpa/* netinet/* bits/* security/* asm/* gnu/* linux/* gentoo* | tee -a $LOG
   cd /usr/include/linux; h2ph * | tee -a $LOG
}


# Build a list of modules installed under older perls - only valid if the module was an ebuild :)
module_list() {
   echo ""
   echo "$(date) : Building list of modules for reinstall" | tee -a $LOG
   echo "Locating modules for reinstall"
    for checkfile in `find $PKGDIR -maxdepth 3 -mindepth 3 -name "CONTENTS" |xargs grep -El '/usr/lib*/perl5/' `; do
   if [ "`grep -l "${gPERL_VERSION}" $checkfile`x" = "x" ]; then
       echo "$checkfile" >> ${MODULES_LIST}
   fi;
    done
}

alternate_module_list() {
   # Takes longer to run, but identifes modules not associated with
   # an ebuild.
   #
   # Reset INC - INC is dynamically generated, and if we removed any ph
   # files - and they were the only thing left in a dir - then there's
   # no sense in revisiting that dir
   echo ""
   echo "$(date) : Building list of modules for reinstall" | tee -a $LOG
   INC=$(perl -e 'for $line (@INC) { next if $line eq "."; next if $line =~ m/'${PERL_VERSION}'/; print "$line\n" }')
   INSTALLED_EBUILDS=$(find $PKGDIR -name CONTENTS)
   echo "Locating modules for reinstall"
   for file in $(find $INC -iname "*.pm" \! -type f 2>/dev/null | fgrep -v "${gPERL_VERSION}"| sort -u); do
      PKG=$(for ebuild in $INSTALLED_EBUILDS; do if fgrep -l $file $ebuild; then break; fi; done)
      if [ -z "$PKG" ] ; then
         echo "Warning: $file is not owned by a currently installed ebuild"
      else
         echo "$PKG" >>${MODULES_LIST}
      fi
   done
}

# ebuild_find searches the repositories for ebuilds
# note the ! in front of paludis to get the same
# return state as the emerge/grep combo
ebuild_find() {
    if [ -z $PALUDIS ]; then
        emerge --oneshot -p $1|egrep -q ".*ebuilds.*satisfy"
    else
        ! paludis -q $1 --no-suggestions --compact > /dev/null 2>&1
    fi
    STATUS=$?
    return $STATUS
}

# The meat of it - rebuilding the ebuilds
# ALL emerges are oneshots - we don't want to mess with the world file
# We first attempt to emerge the specific module that was installed last time
# If that fails, we attempt to install a newer version

ebuild_rebuild() {

   echo ""
   echo "$(date) : Rebuilding modules: Building list of ebuilds" | tee -a $LOG
   if [ -s ${MODULES_LIST} ]; then
      for line in $(sort -u ${MODULES_LIST}); do
         echo "$line"|sed -e 's|.*pkg/||' -e 's|/CONTENTS||'|fgrep -v "dev-lang/perl" >>${EBUILDS_PREINSTALL}
      done
   fi

   # If they asked for interactive, let them see what will be reinstalled
   if [ -s ${EBUILDS_PREINSTALL} ]; then

      if [ ! -z $ASK ]; then
         echo "Press Enter to see the list of ebuilds we'll be evaluating"
         read key
         $PAGER ${EBUILDS_PREINSTALL}
         printf "Continue? (Y/N) "
         read ANSWER
         if [ $(echo "${ANSWER}" | grep -i -e "^n|N" ) ]; then
            echo "$(date) : USER ABORTED REBUILD">>$LOG
            exit
         fi
      fi

      for EBUILD in $(cat ${EBUILDS_PREINSTALL} ); do
# Use the esync cache if available
         if [ -z $PALUDIS ] && [ -x /usr/bin/esearch ]; then
            EBUILD=`echo $EBUILD | sed -e 's/-[0-9].*//'`
            INFO=$(/usr/bin/esearch --fullname --instonly --own='%p:%vi:%va:%m' $EBUILD)
            #FULLNAME=`echo $INFO | cut -d':' -f1`
            INSTALLED=`echo $INFO | cut -d':' -f2`
            AVAILABLE=`echo $INFO | cut -d':' -f3`
            MASKED=`echo $INFO | cut -d':' -f4`
            if [ ! -z "$MASKED" ]; then
               echo ""
               echo "$(date) : There are no unmasked ebuilds to satisfy $EBUILD. Skipping" | tee -a $LOG
               sleep 2
            else
               if [ "$INSTALLED" != "$AVAILABLE" -a -n "$ASK" ]; then
                  printf "${EBUILD}-${INSTALLED} is not the latest available. Use version ${AVAILABLE}? (Y/n) "
                  read ANSWER
                  if [ $(echo "${ANSWER}" | grep -i "^n" ) ]; then
                     # re-install the current version
                     echo "=${EBUILD}-${INSTALLED}" >> ${EBUILDS_ORDERED}
                  else
                     # we want the latest available
                     echo "$EBUILD" >> ${EBUILDS_ORDERED}
                     echo "$(date) : User chose to install ${EBUILD}-${AVAILABLE}" >> $LOG
                  fi
               else
                  # assume we want the latest available
                  echo "$EBUILD" >> ${EBUILDS_ORDERED}
               fi
            fi
         else
# No cache available - use the old method
         if ebuild_find "=$EBUILD"; then
            if ebuild_find ">=$EBUILD"; then
               echo "$(date) : There are no unmasked ebuilds to satisfy $EBUILD. Skipping" | tee -a $LOG
               sleep 2
            else
               if [ ! -z $ASK ]; then
                  printf "${EBUILD} isn't available, but a new version is. Install? (Y/N) "
                  read ANSWER
                  if [ $(echo "${ANSWER}" | egrep -e "^y|Y" ) ]; then
                     echo ">=$EBUILD" >> ${EBUILDS_ORDERED}
                     echo "$(date) : User chose to install >=${EBUILD}">>$LOG
                  fi
               else
                  echo ">=$EBUILD" >>${EBUILDS_ORDERED}
               fi
            fi
         else
            echo "=$EBUILD">>${EBUILDS_ORDERED}
         fi
         fi
      done

      if [ -s ${EBUILDS_ORDERED} ]; then
         if [ ! -z $ASK ]; then
            echo "Press Enter to see the final list of ebuilds to install"
            read key
            $PAGER ${EBUILDS_ORDERED}
            printf "Continue? (Y/N) "
            read ANSWER
            if [ $(echo "${ANSWER}" | egrep -e "^n|N" ) ]; then
               echo "$(date) : USER ABORTED REBUILD">>$LOG
               exit
            fi
         fi

    if [ -z $PALUDIS ]; then
             # Cut down to one line so portage can handle ordering these appropriately
             emerge -pq --oneshot $(cat ${EBUILDS_ORDERED}) | sed -n -e 's/^\[ebuild .*\] \([^ ]*\).*/=\1/p' >>${EBUILDS_REINSTALL}

             echo ""
             echo "Reinstalling ebuilds"
             echo "$(date) : Ebuilds to reinstall: ">>$LOG
             cat ${EBUILDS_REINSTALL}>>$LOG
             echo >>$LOG

             # Now that we have them in the right order, emerge them one at a time
             # This is to avoid problems if one doesn't emerge correctly

             for EBUILD in $(cat ${EBUILDS_REINSTALL}); do
       emerge --oneshot ${EMERGE_OPTIONS} "$EBUILD"
             done
    else
        # Paludis does ordering and handle spurious errors
        paludis -i1 --continue-on-failure always ${PALUDIS_OPTIONS} $(cat ${EBUILDS_ORDERED})
    fi
      else
         echo
         echo "Nothing to reinstall!"
         echo
      fi
   else
      echo
      echo "Nothing to reinstall!"
      echo
   fi

}

# Locate .so's and binaries linked against libperl.so
# The coup is in ! -newer libperl.so - cut out anything that was obviously installed
# after our last install of libperl, which should cut out the false positives.

libperl_list() {
   echo ""
   echo "$(date) : Locating ebuilds linked against libperl" | tee -a $LOG
   for i in $(find $(egrep -v "^#" /etc/ld.so.conf) -type f -name '*.so*' ! -newer /usr/lib/libperl.so 2>/dev/null) \
            $(find $(echo $PATH | sed 's/:/ /g') -type f -perm +0111 ! -newer /usr/lib/libperl.so 2>/dev/null) ;
   do
      if [ -f ${i} ]; then
         if ldd ${i} 2>&1 | fgrep "libperl" - >/dev/null; then
            for file in $PKGDIR/*/*/CONTENTS; do
               fgrep -l " $i " $file
            done >>${MODULES_LIST};
         fi   
      fi
   done

}

# Assuming a successful module run, look to see whats left over
leftovers() {
   echo ""
   echo "$(date) : Finding left over modules" | tee -a $LOG

   echo ""
   echo "$(date) : The following files remain. These were either installed by hand" | tee -a $LOG
   echo "$(date) : or edited. This script cannot deal with them." | tee -a $LOG
   echo | tee -a $LOG


   INC=$(perl -e 'for $line (@INC) { next if $line eq "."; next if $line =~ m/'${PERL_VERSION}'/; print "$line\n" }')
   for file in $(find $INC -type f 2>/dev/null |fgrep -v  "${gPERL_VERSION}" ) ; do
      echo "$(date) : ${file}" | tee -a $LOG
   done
}

usage() {
   echo "Usage: $0 [options] [paludis] [ask]"
   printf "\tmodules - rebuild perl modules for old installs of perl\n"
   printf "\tallmodules - rebuild perl modules for any install of perl\n"
   printf "\tlibperl - rebuild anything linked against libperl\n"
   printf "\tph-clean - clean out old ph files from a previous perl\n"
   printf "\tphupdate - update existing ph files, useful after an upgrade to system parts like the kernel\n"
   printf "\tphall - clean out old ph files and run phupdate\n"
   printf "\tall - rebuild modules, libperl linkages, clean ph files, and rebuild them\n"
   printf "\treallyall - rebuild modules for any install of perl, libperl linkages, clean ph files, and rebuild them\n"
   printf "\n"
   printf "\task - ask for confirmation on each emerge\n"
   printf "\tpaludis - use the paludis package mangler\n"
   printf "\n"
   exit 0
}

if [ -z "$1" ]; then
   usage
fi

EMERGE_OPTIONS=""
PALUDIS_OPTIONS=""
ASK=""
PALUDIS=""
MODULES=false
LIBPERL=false
PHCLEAN=false
PHUPDATE=false
FORCE=false
LEFTOVERS=false
while [ ! -z "$1" ] ; do
   case "$1" in
      help | --help | -h )
         usage
         ;;
      leftovers )
         LEFTOVERS=true
         shift
         ;;
      modules )
         MODULES=true
#         LEFTOVERS=true
         shift
         ;;
      allmodules )
         MODULES=true
         FORCE=true
         shift
         ;;
      libperl )
         LIBPERL=true
         shift
         ;;
      paludis )
    PALUDIS="true"
    shift
    ;;
      ph-clean )
         PHCLEAN=true
         shift
         ;;
      phupdate )
         PHUPDATE=true
         shift
         ;;
      phall )
         PHCLEAN=true
         PHUPDATE=true
         shift
         ;;
      all )
         MODULES=true
         LIBPERL=true
         PHCLEAN=true
         PHUPDATE=true
         LEFTOVERS=true
         shift
         ;;
      reallyall )
         MODULES=true
         LIBPERL=true
         PHCLEAN=true
         PHUPDATE=true
         FORCE=true
   shift
         ;;
      ask )
         ASK="true"
         EMERGE_OPTIONS="${EMERGE_OPTIONS} --ask"
         shift
         ;;
      force )
         FORCE=true
         shift
         ;;
      * )
         EMERGE_OPTIONS="${EMERGE_OPTIONS} $1"
         PALUDIS_OPTIONS="${PALUDIS_OPTIONS} $1"
         shift
         ;;
   esac
done

if [ ! -z $PALUDIS ];then
    PKGDIR=$(paludis --configuration-variable installed location)
else
    PKGDIR=$(/usr/bin/portageq vdb_path)
fi

$FORCE && PERL_VERSION="0.0.0" && gPERL_VERSION="0\.0\.0"
$PHCLEAN && ph_clean
$PHUPDATE && ph_update
$MODULES && module_list
$LIBPERL && libperl_list
($MODULES || $LIBPERL) && ebuild_rebuild
$LEFTOVERS && leftovers

#postclean

exit
# vim:ts=3:sw=3:et



Remember that it is NOT extensively tested (after all, the reason I'm posting here is to get it tested).


Last edited by Styrsven on Sun Sep 06, 2009 4:57 pm; edited 1 time in total
Back to top
View user's profile Send private message
yoshi314
l33t
l33t


Joined: 30 Dec 2004
Posts: 848
Location: PL

PostPosted: Mon Aug 31, 2009 7:42 pm    Post subject: Reply with quote

can you post the script for download (e.g. pastebin) ? i have syntax error on line 293
_________________
~amd64
shrink your /usr/portage with squashfs+aufs
Back to top
View user's profile Send private message
Styrsven
n00b
n00b


Joined: 30 Aug 2009
Posts: 5

PostPosted: Mon Aug 31, 2009 8:06 pm    Post subject: Reply with quote

yoshi314 wrote:
can you post the script for download (e.g. pastebin) ? i have syntax error on line 293


I hope this is better...

http://pastebin.com/f1fe8af93
Back to top
View user's profile Send private message
Styrsven
n00b
n00b


Joined: 30 Aug 2009
Posts: 5

PostPosted: Sun Sep 06, 2009 4:59 pm    Post subject: Reply with quote

Updated first post.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Unsupported Software All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum