Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Putting all installed packages into world-file (update ALL p
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
pZYchO
n00b
n00b


Joined: 28 Apr 2003
Posts: 61

PostPosted: Tue Apr 20, 2004 9:10 pm    Post subject: Putting all installed packages into world-file (update ALL p Reply with quote

Hi everyone,

I often had the problem, that not all packages I have installed were patched. Even the often mentioned -D switch didn't work.
The problem is, that "emerge -uD world" doesn't check all installed packages (packages which were installed as dependency aren't in world-file).
I have not much experience with portage so feel free to correct me... =)

I wrote a (quick and dirty) script which looks up ALL installed packages and creates a patched world-file:

Code:

#!/bin/bash
DSTDIR=`pwd`
if [ $DSTDIR = '/var/cache/edb' ]; then
        echo "Never start this script in /var/cache/edb!"
        exit 1
fi
test -e $DSTDIR/installed_packages && rm $DSTDIR/installed_packages
#
cd /var/db/pkg
for i in `find . -type d -maxdepth 2 -mindepth 2 | sed 's/\.\///g'`; do
        curpackage=`ACCEPT_KEYWORDS="~x86 x86" emerge -p =$i --nospinner --columns | grep ebuild | awk -F\] '{ print $2 }' | awk -F\[ '{ print $1 }' | sed s/\ //g`
        c=0
        if [ -e $DSTDIR/installed_packages ]; then
                for j in `cat $DSTDIR/installed_packages`; do
                        if [ $j = $curpackage ]; then
                                c=$(($c+1))
                        fi
                done
        fi
        if [ $c -eq 0 ]; then
                echo $curpackage >> $DSTDIR/installed_packages
        fi
done
#
cd $DSTDIR
cp /var/cache/edb/world .
#
diff -u --minimal world installed_packages | patch world


The patched world-file can be copied to /var/cache/edb/ so portage will check all packages if you run "emerge -u world".

If you don't like to edit your world-file you can do like this

Code:

for i in `cat installed_packages`; do
        echo $i
        emerge -u $i --nospinner -p
done


to check for updates.

I tested the script several time, but I'm not sure if it is 100% sure.

I hope someone knows a better way to solve that problem, but I haven't found one jet...

Greetz pZYchO

P.S.: Sorry for my bad english, I'm german...


Last edited by pZYchO on Wed Apr 21, 2004 9:17 am; edited 1 time in total
Back to top
View user's profile Send private message
joycea
Apprentice
Apprentice


Joined: 23 Jul 2002
Posts: 191
Location: Kingston, Canada

PostPosted: Tue Apr 20, 2004 9:41 pm    Post subject: Reply with quote

As I understand it, Gentoo calculates reverse dependencies based on what is not in your world file. That is, you only put the package you are interested in into the world file and then the dependencies follow from this. So, it is not a problem to add everything in if you are just installing and forgetting, but if you ever want to remove a package and all it's dependencies it's going to be rough if you have replaced the world file.

Just some food for thought, based on this your second method seems more 'Gentoo-ish' in approach.
Back to top
View user's profile Send private message
pZYchO
n00b
n00b


Joined: 28 Apr 2003
Posts: 61

PostPosted: Wed Apr 21, 2004 5:42 am    Post subject: Reply with quote

That's correct. I also prefer the second way.

But that's the only way I found to update ALL packages. I hope someone got a better one, 'cause the mass of emerges take a long time... :(


Last edited by pZYchO on Wed Apr 21, 2004 7:37 am; edited 1 time in total
Back to top
View user's profile Send private message
jkt
Retired Dev
Retired Dev


Joined: 06 Feb 2004
Posts: 1250
Location: Prague, Czech republic, EU

PostPosted: Wed Apr 21, 2004 6:55 am    Post subject: Reply with quote

are you sure that `emerge -uD world` doesn't work and your solution does? in other words, are you sure or is it just your impression? i'm using `emerge --update --pretend --verbose --deep world` (ie. `emerge -uDpv world`) and it seems to be ok...
Back to top
View user's profile Send private message
pZYchO
n00b
n00b


Joined: 28 Apr 2003
Posts: 61

PostPosted: Wed Apr 21, 2004 7:34 am    Post subject: Reply with quote

I am definitly sure...
Here an example (that's why I wrote the script):

Code:

myserver / # emerge -uDp --nospinner world

These are the packages that I would merge, in order:

Calculating world dependencies  ...done!

myserver / # emerge -u mod_php -p --nospinner

These are the packages that I would merge, in order:

Calculating dependencies  ...done!
[ebuild     U ] dev-libs/libxslt-1.1.3 [1.1.1]
[ebuild     U ] media-libs/freetype-2.1.5-r1 [2.1.4]
[ebuild     U ] net-libs/libwww-5.4.0-r2 [5.4.0-r1]
[ebuild  N    ] media-libs/pdflib-5.0.2
[ebuild     U ] net-misc/curl-7.10.8-r1 [7.10.7]
[ebuild  N    ] media-libs/t1lib-5.0.0-r2
[ebuild  N    ] dev-php/mod_php-4.3.6_rc2
[ebuild  N    ] dev-php/php-4.3.4-r4

(mod_php is installed, I don't know why emerge want to install it instead of updating it.)

I also thought, that emerge -uD world updates ALL packages, but it doesn't. I don't know if my solution is the best, but it works.
All packages you have installed can be found in /var/db/pkg. Also those which are installed as dependency. That is my source (find . -maxdepth 2...). But the directory names contain the version number, that's why I used the emerge -p =$i and so on.
My next step is to check if the package is already in my list. This is important to avoid duplicates (if you have 2 different kernel versions merged for example).
The file "installed_packages" contains all packages installed, after a diff with your world file you can patch it with all packages not in world file.
This is like joycea allready mentioned not always a good idea, that's why I postet the second way to patch all packages without touching the world-file...

Hope you understand my english (remember, I'm german... =))
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 18154

PostPosted: Wed Apr 21, 2004 7:09 pm    Post subject: Reply with quote

Moved from Installing Gentoo.
_________________
Those who know what's best for us must rise and save us from ourselves.
Back to top
View user's profile Send private message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2870
Location: Bay Area, CA

PostPosted: Thu Apr 22, 2004 5:32 am    Post subject: Reply with quote

pZYchO wrote:
I am definitly sure...
Here an example (that's why I wrote the script):

Code:

myserver / # emerge -uDp --nospinner world

These are the packages that I would merge, in order:

Calculating world dependencies  ...done!

myserver / # emerge -u mod_php -p --nospinner

These are the packages that I would merge, in order:

Calculating dependencies  ...done!
[ebuild     U ] dev-libs/libxslt-1.1.3 [1.1.1]
[ebuild     U ] media-libs/freetype-2.1.5-r1 [2.1.4]
[ebuild     U ] net-libs/libwww-5.4.0-r2 [5.4.0-r1]
[ebuild  N    ] media-libs/pdflib-5.0.2
[ebuild     U ] net-misc/curl-7.10.8-r1 [7.10.7]
[ebuild  N    ] media-libs/t1lib-5.0.0-r2
[ebuild  N    ] dev-php/mod_php-4.3.6_rc2
[ebuild  N    ] dev-php/php-4.3.4-r4

(mod_php is installed, I don't know why emerge want to install it instead of updating it.)

I also thought, that emerge -uD world updates ALL packages, but it doesn't. I don't know if my solution is the best, but it works.

Hope you understand my english (remember, I'm german... =))

very interesting observation there...This is most likely a bug(or may be a feature) in portage. For example, I have openssl installed but its not present in world file and its not reported as "-N". It was recently updated as part of the "emerge -uD world". Do you see a directory /var/db/pkg/dev-php/php-4.x.y with CONTENTS file in it?

And btw, your english is just fine...
Back to top
View user's profile Send private message
pZYchO
n00b
n00b


Joined: 28 Apr 2003
Posts: 61

PostPosted: Thu Apr 22, 2004 8:49 am    Post subject: Reply with quote

No, I haven't. I tried curl only, to avoid that php is the problem (cause it is not installed but will be installed if I run emerge -u mod_php). The problem is the same:

Code:

myserver / # emerge -uDp --nospinner world

These are the packages that I would merge, in order:

Calculating world dependencies  ...done!

myserver / # emerge -uD --nospinner curl -p

These are the packages that I would merge, in order:

Calculating dependencies  ...done!
[ebuild     U ] net-misc/curl-7.10.8-r1 [7.10.7]

I also had cases where packages were updates which were not in my world file. But I can't figure out the difference between these packages...

Btw, the output of the second way is better this way:
Code:

for i in `cat installed_packages`; do
        emerge -u $i --nospinner -p | grep -i ebuild
done


Greetz
Back to top
View user's profile Send private message
heeckhau
n00b
n00b


Joined: 19 Mar 2003
Posts: 31
Location: Gent, Belgium

PostPosted: Sat Apr 24, 2004 7:34 pm    Post subject: Reply with quote

Shouldn't you just run the regenworld script ?
Code:
regenworld
Back to top
View user's profile Send private message
pZYchO
n00b
n00b


Joined: 28 Apr 2003
Posts: 61

PostPosted: Sun Apr 25, 2004 11:26 am    Post subject: Reply with quote

That seems to be the "nice" solution. I'll try this on all systems. Perhaps that's the solution I was looking for, thx.
Is there any documentation?
Back to top
View user's profile Send private message
Narada
Guru
Guru


Joined: 12 Dec 2002
Posts: 300
Location: London, UK

PostPosted: Sun Apr 25, 2004 11:49 am    Post subject: Reply with quote

pZYchO wrote:
That seems to be the "nice" solution. I'll try this on all systems. Perhaps that's the solution I was looking for, thx.
Is there any documentation?


Regenworld checks all installed packages and regenerates the world file and overwrites the old one. It also shows you any additions that it made. There isn't any documentation as such that I know of. Periodically, running it will ensure your world file is kept up to date with all packages you installed manually.
_________________
http://dhruba.name/
Back to top
View user's profile Send private message
tomk
Bodhisattva
Bodhisattva


Joined: 23 Sep 2003
Posts: 7221
Location: Sat in front of my computer

PostPosted: Sun Apr 25, 2004 12:18 pm    Post subject: Reply with quote

pZYchO wrote:
Is there any documentation?


There's a FAQ about it in the FAQ forum:

https://forums.gentoo.org/viewtopic.php?t=136627

It also has a one line command which does what your script does, using qpkg.
_________________
Search | Read | Answer | Report | Strip
Back to top
View user's profile Send private message
pZYchO
n00b
n00b


Joined: 28 Apr 2003
Posts: 61

PostPosted: Sun Apr 25, 2004 8:38 pm    Post subject: Reply with quote

I was looking for that a long time.
Thx.
Back to top
View user's profile Send private message
jkt
Retired Dev
Retired Dev


Joined: 06 Feb 2004
Posts: 1250
Location: Prague, Czech republic, EU

PostPosted: Tue Apr 27, 2004 2:11 pm    Post subject: Reply with quote

pZYchO:
have you emerged the package which isn't in your world file in "normal" way, ie. the _latest_ version? I'venoticed that when merging something like `emerge =pkgname-pkgversion`, I haven't the new package recorded in the world file... bug or feature ;-) ?
Back to top
View user's profile Send private message
pZYchO
n00b
n00b


Joined: 28 Apr 2003
Posts: 61

PostPosted: Wed Apr 28, 2004 8:10 am    Post subject: Reply with quote

It not a bug, it's a feature... (normally a m$ slogan... =))

I think that portage shouldn't add the package to your world file, 'cause it detects that you have it allready installed and updates the package. When portage updates a package it doesn't add it to the world file.......
Back to top
View user's profile Send private message
jkt
Retired Dev
Retired Dev


Joined: 06 Feb 2004
Posts: 1250
Location: Prague, Czech republic, EU

PostPosted: Fri Apr 30, 2004 10:50 am    Post subject: Reply with quote

pZYchO wrote:
When portage updates a package it doesn't add it to the world file.......


I was talking about installing a package for the first time, not about upgrading ;-)
Back to top
View user's profile Send private message
pZYchO
n00b
n00b


Joined: 28 Apr 2003
Posts: 61

PostPosted: Fri Apr 30, 2004 12:02 pm    Post subject: Reply with quote

Sorry... I misunderstood...

I don't know why this happens, perhaps that happend on my server, too. But regenworld fixed it... =) I run it every time I emerge something new. Hopefully problems like the above will not appear anymore...

Greetz...
Back to top
View user's profile Send private message
mgsnova
n00b
n00b


Joined: 24 Apr 2004
Posts: 7
Location: Germany

PostPosted: Sat May 01, 2004 3:34 pm    Post subject: Reply with quote

Great work but running the Script i get this:

./script: line 15: [: sys-kernel/gentoo-dev-sources: unary operator expected
and so on ... this happens not to all packages ... ?!

but anyway, after little seraching i tried this: 'qpkg -I -nc > installed_packages'. It does the same like the script ;-)

mgsnova
Back to top
View user's profile Send private message
pZYchO
n00b
n00b


Joined: 28 Apr 2003
Posts: 61

PostPosted: Tue May 04, 2004 1:36 pm    Post subject: Reply with quote

I know this but. Here is my final Version. But anyway, like you already mentioned, this script is obsolete... =)

Code:

#!/bin/bash
cd /home/root/jobs
DSTDIR=`pwd`
if [ $DSTDIR != '/home/root/jobs' ]; then
        echo "Start this script from /home/root/jobs!"
        exit 1
fi
test -e $DSTDIR/installed_packages && rm $DSTDIR/installed_packages
#
cd /var/db/pkg
for i in `find . -type d -maxdepth 2 -mindepth 2 | sed 's/\.\///g'`; do
        curpackage=`ACCEPT_KEYWORDS="~x86 x86" emerge -p =$i --nospinner --columns | grep ebuild | awk -F\] '{ print $2 }' | awk -F\[ '{ print $1 }' | sed s/\ //g`
        c=0
        if [ -e $DSTDIR/installed_packages ]; then
                for j in `cat $DSTDIR/installed_packages`; do
                        if [ "$j" = "$curpackage" ]; then
                                c=$(($c+1))
                        fi
                done
        fi
        for j in `cat /var/cache/edb/world`; do
                if [ "$j" = "$curpackage" ]; then
                        c=$(($c+1))
                fi
        done
        if [ $c -eq 0 ]; then
                echo $curpackage >> $DSTDIR/installed_packages
        fi
done
#
cd $DSTDIR
#cp /var/cache/edb/world .
#
#diff -u --minimal world installed_packages | patch world
for i in `cat $DSTDIR/installed_packages`; do
        emerge -up --nospinner $i | grep -i ebuild
done
rm $DSTDIR/installed_packages
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum