Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Scripts to remove old binary packages and source files
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
mark_ar
n00b
n00b


Joined: 17 Nov 2002
Posts: 13
Location: Madison, WI

PostPosted: Sun Nov 17, 2002 6:14 pm    Post subject: Scripts to remove old binary packages and source files Reply with quote

Description:
The following two scripts will remove old binary packages and source files from the portage tree. On my system, this eliminated over 1 GB of files.

Advantages:
As far as I 've seen, other cleaning scripts do not query portage for a list of installed packages. Consequently, older but still useful/needed binary packages and source files are removed from the portage tree. These scripts will keep all binary packages and source files belonging to packages that are installed on the system. Everything else is removed.

** README - IMPORTANT **
The scripts require the app-admin/gentoolkit package to be installed. Because the scripts modify the portage tree, they must be run as root.

In the clean_packages.sh script, you must set the variables package_folder and temp_folder. package_folder is the location of binary portage packages (normally /usr/portage/packages in the default configuration). temp_folder is a temporary folder used for storing binary packages while the script is running. In the clean_sources.sh script, you must set portage_folder and temp_folder. portage_folder is the location of the portage tree (normally /usr/portage in the default configuration). The script assumes that source files are found in the directory $portage_folder/distfiles.

Finally, these scripts perform NO error checking, so consider yourself warned. They work correctly on my machine and configuration, but I can't promise the same on yours.

clean_packages.sh
Code:

# !/bin/bash
# clean_packages.sh
# Remove old binary packages from portage

package_folder=/usr/portage/packages
temp_folder=/tmp/temp_packages

cd $package_folder/All
if [ ! -e "$temp_folder" ]
then
   mkdir $temp_folder
else
   rm -rf $temp_folder
   mkdir $temp_folder
fi

packages=$(qpkg -I -v -nc | sort)
for package in $packages
do
   package_name="$(basename $package).tbz2"

   if [ -e "$package_name" ]
   then
      mv $package_name $temp_folder
   fi
done

cd $package_folder
rm -rf *
mkdir $package_folder/All

for package in $packages
do
   package_type=$(dirname $package)
   package_name="$(basename $package).tbz2"

   if [ -e "$temp_folder/$package_name" ]
   then
      mv $temp_folder/$package_name $package_folder/All

      mkdir -p $package_folder/$package_type
      cd $package_folder/$package_type
      ln -s ../All/$package_name $package_name
   fi
done

rm -rf $temp_folder


clean_sources.sh
Code:

# !/bin/bash
# clean_sources.sh
# Remove old source files from portage

portage_folder=/usr/portage
temp_folder=/tmp/temp_sources

if [ ! -e "$temp_folder" ]
then
   mkdir $temp_folder
else
   rm -rf $temp_folder
   mkdir $temp_folder
fi

packages=$(qpkg -I -nc | sort)
for package in $packages
do
   versions=$(qpkg -I -v -nc $package | sort)

   cd $portage_folder/$package/files
   for version in $versions
   do
      digest="digest-$(basename $version)"

      if [ -e $digest ]
      then
         sources=$(cat $digest | cut -d " " -f 3)
         echo -e "$version"

         for source in $sources
         do
            file=$portage_folder/distfiles/$source
            if [ -e $file ]
            then
               echo $file
               mv $file $temp_folder
            fi
         done
         echo
      fi
   done
done

rm -rf $portage_folder/distfiles/*
mv $temp_folder/* $portage_folder/distfiles
rm -rf $temp_folder
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 18165

PostPosted: Sun Nov 17, 2002 7:02 pm    Post subject: Reply with quote

Also of interest might be kerframil's Cleaning out stale distfiles.
_________________
Those who know what's best for us must rise and save us from ourselves.
Back to top
View user's profile Send private message
mark_ar
n00b
n00b


Joined: 17 Nov 2002
Posts: 13
Location: Madison, WI

PostPosted: Mon Nov 18, 2002 1:08 am    Post subject: Reply with quote

kanuslupus wrote:
Also of interest might be kerframil's Cleaning out stale distfiles.


See my comment above describing the advantages of my script. kerframil's script will clean out source files that are still needed. For example, try the command "qpkg -I -v docbook-sgml-dtd" on your system.

On mine, I get the following list:
app-text/docbook-sgml-dtd-3.0 *
app-text/docbook-sgml-dtd-3.1 *
app-text/docbook-sgml-dtd-4.0 *
app-text/docbook-sgml-dtd-4.1 *

My script will keep all packages and source files for those four versions. kerframil's script will happily delete the three oldest ones. His script works, but it is not a complete solution. Additionally, what happens for the package x11-base/xfree-4.2.1? It requires the following files:

X420src-1.tgz
X420src-2.tgz
X420src-3.tgz
freetype-2.1.2.tar.bz2
fcpackage.2_0.tar.gz
4.2.0-4.2.1.diff.gz
XFree86-4.2.1-patches-1.0.tar.bz2
xf86Wacom.c.gz
savage-1.1.25t.tgz
sis_drv_src_251002-2.tar.gz
glide3-headers.tar.bz2
andale32.exe
arial32.exe
arialb32.exe
comic32.exe
courie32.exe
georgi32.exe
impact32.exe
times32.exe
trebuc32.exe
verdan32.exe
webdin32.exe

After running my clean_sources.sh script, if you have xfree-4.2.1 installed, then those files will still be there. In the end, that's why I wrote these scripts. I couldn't find anything that kept ALL the correct files.
Back to top
View user's profile Send private message
jack_mort
Apprentice
Apprentice


Joined: 30 Jan 2003
Posts: 167

PostPosted: Thu Feb 20, 2003 12:37 pm    Post subject: Reply with quote

Wow ! Thanks a lot for your scripts, they work very well :) I was looking for something like that and found this https://forums.gentoo.org/viewtopic.php?t=3011 but as you said they would delete all "old" files even if they're not "old" :S And then I found your script, working perfectly :D

Very good work 8)
Back to top
View user's profile Send private message
smokeslikeapoet
Tux's lil' helper
Tux's lil' helper


Joined: 03 Apr 2003
Posts: 96
Location: Cordova, TN USA

PostPosted: Thu Jun 12, 2003 12:06 am    Post subject: Reply with quote

Oops, I messed up. I don't have my distfiles stored in the default location. I edited the script, but i missed something and ended up deleting all my files. Just a warning to the people who don't store their distfiles under your $/portage directory.
_________________
-SmokesLikeaPoet

Folding@Home
Back to top
View user's profile Send private message
TGL
Bodhisattva
Bodhisattva


Joined: 02 Jun 2002
Posts: 1978
Location: Rennes, France

PostPosted: Thu Jun 12, 2003 9:36 am    Post subject: Reply with quote

Here is a version that doesn't need editing by hand. I've also added some code to protect files from PORTDIR_OVERLAY packages.

clean_sources.sh:
Code:
# !/bin/bash
# clean_sources.sh
# Remove old source files from portage
# http://forums.gentoo.org/viewtopic.php?t=23027

temp_folder=/tmp/temp_sources

portageq=/usr/lib/portage/bin/portageq
portage_folder=`$portageq portdir`
distfiles_folder=`$portageq distdir`
overlay_folder=`$portageq portdir_overlay`

if [ ! -e "$temp_folder" ]
then
   mkdir $temp_folder
else
   rm -rf $temp_folder
   mkdir $temp_folder
fi

packages=$(qpkg -I -nc | sort)
for package in $packages
do
   versions=$(qpkg -I -v -nc $package | sort)

   for folder in $portage_folder $overlay_folder
   do
     if [ -d $folder/$package/files ]
     then
       cd $folder/$package/files
       for version in $versions
       do
          digest="digest-$(basename $version)"
 
          if [ -e $digest ]
          then
             sources=$(cat $digest | cut -d " " -f 3)
             echo -e "$version"
   
             for source in $sources
             do
                file=$distfiles_folder/$source
                if [ -e $file ]
                then
                   echo $file
                   mv $file $temp_folder
                fi
             done
             echo
          fi
       done
     fi
   done
done

rm -rf $distfiles_folder/*
mv $temp_folder/* $distfiles_folder
rm -rf $temp_folder
Back to top
View user's profile Send private message
far
Guru
Guru


Joined: 10 Mar 2003
Posts: 394
Location: Stockholm, Sweden

PostPosted: Thu Jun 12, 2003 8:29 pm    Post subject: Reply with quote

mark_ar wrote:
kanuslupus wrote:
Also of interest might be kerframil's Cleaning out stale distfiles.


See my comment above describing the advantages of my script. kerframil's script will clean out source files that are still needed.

Check that thread again, there are better scripts than kerframil's
_________________
The Porthole Portage Frontend
Back to top
View user's profile Send private message
zakl
n00b
n00b


Joined: 09 May 2003
Posts: 73

PostPosted: Sun Jun 29, 2003 8:07 pm    Post subject: Reply with quote

:cry: I don't know what I did wrong, but I created the clean_packages.sh file in my home directory and forgot to edit the file, and meant to execute the clean_sources.sh file, but executed the packages.sh file instead, and it wiped my home directory T_T

Fair warning, don't forget to edit the files T_T

I don't suppose any of you know of a way to recover deleted files do you?

Zak
Back to top
View user's profile Send private message
Guardian_SkaraBrae
n00b
n00b


Joined: 30 Jul 2003
Posts: 2
Location: Brasil

PostPosted: Sat Aug 09, 2003 3:23 am    Post subject: Reply with quote

hi,

if youdiscover some way to recover your files, please, tell me ...


see ya
Back to top
View user's profile Send private message
siti
Tux's lil' helper
Tux's lil' helper


Joined: 05 May 2003
Posts: 118
Location: Canterbury, New Zealand

PostPosted: Sat Aug 09, 2003 3:36 am    Post subject: Reply with quote

I think you should add a pretend flag so no one wipes out what they don't want. :D
Back to top
View user's profile Send private message
mlybarger
Guru
Guru


Joined: 04 Sep 2002
Posts: 463

PostPosted: Tue Aug 12, 2003 1:35 am    Post subject: Reply with quote

perhaps a little sanity check in the script? i didn't have the gentoolkit installed, and this script happily removed all my files from /usr/portage/distfiles
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum