Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[SOLVED] which files did I edit in /etc ?
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
potuz
Guru
Guru


Joined: 30 Jan 2010
Posts: 378

PostPosted: Thu Feb 13, 2014 5:08 pm    Post subject: [SOLVED] which files did I edit in /etc ? Reply with quote

I have a ton of configuration files that I have edited in under /etc. Typically when I want to install a new system this is the part that gives me most headaches. Is there any way to find out which ones of the files in /etc I edited?

Last edited by potuz on Thu Feb 13, 2014 9:24 pm; edited 1 time in total
Back to top
View user's profile Send private message
aCOSwt
Bodhisattva
Bodhisattva


Joined: 19 Oct 2007
Posts: 2537
Location: Hilbert space

PostPosted: Thu Feb 13, 2014 5:20 pm    Post subject: Reply with quote

I presume that you also regularly etc-update or equivalent when portage requires you to do so, so there is nothing actually relevant to sort based on timestamps ?

If so I do not get any answer for you.
In order to avoid being in the situation you are experiencing, I get a shadow /etc directory in which I put all the config files that I customize. Official /etc entries symlinking to these.
(Of course the shadow directory is on the same filesystem than /etc)
_________________
Back to top
View user's profile Send private message
potuz
Guru
Guru


Joined: 30 Jan 2010
Posts: 378

PostPosted: Thu Feb 13, 2014 6:07 pm    Post subject: Reply with quote

aCOSwt wrote:
I presume that you also regularly etc-update or equivalent when portage requires you to do so, so there is nothing actually relevant to sort based on timestamps ?

If so I do not get any answer for you.
In order to avoid being in the situation you are experiencing, I get a shadow /etc directory in which I put all the config files that I customize. Official /etc entries symlinking to these.
(Of course the shadow directory is on the same filesystem than /etc)

Thanks, that's something very similar to what I used to do many years ago and I should've continued doing, but I guess now it's late :)

This seems to be a very basic feature that a distro might want to offer, specially one like Gentoo... the only important things in my system is the data in /home and the customizations in /etc and in ~/ (which anyway gets backuped with /home). I figured someone must've come up with some sort of script. In principle I could go over the list of installed packages, Check which ones install something in /etc and then check if the current file differs.. but that seems brutal.
Back to top
View user's profile Send private message
dambacher
Apprentice
Apprentice


Joined: 11 Feb 2003
Posts: 288
Location: Germany

PostPosted: Thu Feb 13, 2014 6:16 pm    Post subject: Reply with quote

When portage merges a package it creates checksums into
Code:
/var/db/pkg/${CATEGORY}/${PN}-${PVR}/CONTENTS


You can check these chhecksums with the equery tool if you give it the package name:
Code:
equery check $PACKAGE_NAME


If you want to know wich packge is responsible for a given file in etc you can do
Code:
equery  belongs $FILE


equery is part of app-portage/gentoolkit.
Back to top
View user's profile Send private message
dambacher
Apprentice
Apprentice


Joined: 11 Feb 2003
Posts: 288
Location: Germany

PostPosted: Thu Feb 13, 2014 7:01 pm    Post subject: Reply with quote

i have some script magic for you.
This finds all files you installed in etc, gets their checksums from portage, verifies them and spits you out what was changed and what not:

Code:

find /var/db/pkg/ -name "CONTENTS" -exec cat \{} \; | gawk "/obj\s\/etc/ { print \$3,\$2}"| md5sum -c |less


and if you are only interested in changes:

Code:

find /var/db/pkg/ -name "CONTENTS" -exec cat \{} \; | gawk "/obj\s\/etc/ { print \$3,\$2}"| md5sum -c | grep -v "OK" | sort | less


Keep in mind that this does not detect files you create on your own.
but with another magic you could find the files not registered in CONTENTS, too
Back to top
View user's profile Send private message
potuz
Guru
Guru


Joined: 30 Jan 2010
Posts: 378

PostPosted: Thu Feb 13, 2014 9:24 pm    Post subject: Reply with quote

Thanks, this gets the work done. The problem is again my lazynes that many times on etc-update I disregard updates that seem vacuous and this magic finds of course that the checksum is not like the one in /var/db. For that there's no solution but to actually do the update :) Thanks again!
Back to top
View user's profile Send private message
khayyam
Watchman
Watchman


Joined: 07 Jun 2012
Posts: 6228
Location: Room 101

PostPosted: Fri Feb 14, 2014 12:38 am    Post subject: Reply with quote

dambacher wrote:
Code:
find /var/db/pkg/ -name "CONTENTS" -exec cat \{} \; | gawk "/obj\s\/etc/ { print \$3,\$2}"| md5sum -c | grep -v "OK" | sort | less

dambacher ... UUoC ... also with single quotes there is no need to protect '$', etc.

Code:
find /var/db/pkg/ -name "CONTENTS" -exec awk '/^obj[ ]\/etc/{print $3,$2}' {} \; |md5sum -c | grep -v "OK" | sort | less

best ... khay
Back to top
View user's profile Send private message
dambacher
Apprentice
Apprentice


Joined: 11 Feb 2003
Posts: 288
Location: Germany

PostPosted: Fri Feb 14, 2014 6:44 am    Post subject: Reply with quote

yes you're right, shorter is possible, UUoC may be valid, thanks for the award
but with cat it's faster!

using time -
my version with cat:
Code:

real   0m0.632s
user   0m0.212s
sys   0m0.106s


your code:
Code:

real   0m0.989s
user   0m0.198s
sys   0m0.126s


timed after multiple executions to prevent influence of cache.
Back to top
View user's profile Send private message
khayyam
Watchman
Watchman


Joined: 07 Jun 2012
Posts: 6228
Location: Room 101

PostPosted: Fri Feb 14, 2014 6:02 pm    Post subject: Reply with quote

dambacher wrote:
[...] but with cat it's faster!

dambacher ... I can't compile sys-process/time for some reason so I have no way to compare the two locally (not that time qualifies as a benchmark). Anyhow, if your arguing that 'cat |' will somehow access the files 'faster' (which is what it boils down to) then you also recieve an entropy award.

best ... khay
Back to top
View user's profile Send private message
aCOSwt
Bodhisattva
Bodhisattva


Joined: 19 Oct 2007
Posts: 2537
Location: Hilbert space

PostPosted: Fri Feb 14, 2014 10:30 pm    Post subject: Reply with quote

Out of topic but : Happy to see you back khay!
_________________
Back to top
View user's profile Send private message
dambacher
Apprentice
Apprentice


Joined: 11 Feb 2003
Posts: 288
Location: Germany

PostPosted: Sat Feb 15, 2014 7:49 am    Post subject: Reply with quote

The overhead is simply due to options processing: starting 1126 (numer of installed ebuilds on my system) times cat must be faster than 1126 times awk.
Back to top
View user's profile Send private message
khayyam
Watchman
Watchman


Joined: 07 Jun 2012
Posts: 6228
Location: Room 101

PostPosted: Sat Feb 15, 2014 8:31 am    Post subject: Reply with quote

dambacher wrote:
The overhead is simply due to options processing: starting 1126 (numer of installed ebuilds on my system) times cat must be faster than 1126 times awk.

dambacher ... "must" ... you seem convinced, and what "options processing" are you talking about? As I said it boils down to whether "cat file | awk" is "faster" than "awk file" and if this is the case what kind of special powers of file access does cat possess?

best ... khay

ps: please avoid repling with a similarly sounding "explanation" to the one above, ie, an explanation that simply states it as true ... without providing an actual explanation.
Back to top
View user's profile Send private message
dambacher
Apprentice
Apprentice


Joined: 11 Feb 2003
Posts: 288
Location: Germany

PostPosted: Sat Feb 15, 2014 9:01 am    Post subject: Reply with quote

khay
I'm not arguing that cat $file | awk is faster than awk $file. here you are right.

my point is something else
with my implementation find fires up cat 1126 times to gather data to stdout and then awk once to filter and format.
with your implementation find fires up awk 1126 times directly, each time awk has to process it's command line to set up its filtering
Back to top
View user's profile Send private message
khayyam
Watchman
Watchman


Joined: 07 Jun 2012
Posts: 6228
Location: Room 101

PostPosted: Sat Feb 15, 2014 2:28 pm    Post subject: Reply with quote

dambacher wrote:
[...] with my implementation find fires up cat 1126 times to gather data to stdout and then awk once to filter and format. with your implementation find fires up awk 1126 times directly, each time awk has to process it's command line to set up its filtering

dambacher ... your absolutely correct, that would make a difference. I was competely overlooking the numbers of files involved ... doh!

aCOSwt wrote:
Out of topic but : Happy to see you back khay!

... but I'm not, at least not enough to make you truely happy :)

best ... khay
Back to top
View user's profile Send private message
cwr
Veteran
Veteran


Joined: 17 Dec 2005
Posts: 1969

PostPosted: Sat Feb 15, 2014 2:28 pm    Post subject: Reply with quote

Too late to help you now, but generally the first thing I do on a new install,
after unpacking the stage 3 file and portage snapshot, is to emerge Mercurial
and run hg on the /etc directory.

Once the first fury of the install is over, and I have a working desktop, I run
it again.

Will
Back to top
View user's profile Send private message
John R. Graham
Administrator
Administrator


Joined: 08 Mar 2005
Posts: 10261
Location: Somewhere over Atlanta, Georgia

PostPosted: Sat Feb 15, 2014 6:53 pm    Post subject: Reply with quote

khayyam wrote:
dambacher wrote:
[...] with my implementation find fires up cat 1126 times to gather data to stdout and then awk once to filter and format. with your implementation find fires up awk 1126 times directly, each time awk has to process it's command line to set up its filtering

dambacher ... your absolutely correct, that would make a difference. I was completely overlooking the numbers of files involved ...
The best answer is to launch neither one 1126 (or however many) times. This is exactly what xargs (part of sys-apps/findutils, like find) is for:
Code:
find /var/db/pkg/ -name "CONTENTS" | xargs gawk '/obj\s\/etc/ { print $3,$2}' | md5sum -c | grep -v "OK" | sort
This version launches cat zero times and gawk once (or a very few times) and is faster still. (I think khay's UUoC award stands.)

Happy to see you as well, khay. :)

- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters.
Back to top
View user's profile Send private message
dambacher
Apprentice
Apprentice


Joined: 11 Feb 2003
Posts: 288
Location: Germany

PostPosted: Sat Feb 15, 2014 10:25 pm    Post subject: Reply with quote

John wrote:
Quote:
This version launches cat zero times and gawk once (or a very few times) and is faster still. (I think khay's UUoC award stands.)

xargs was not in my repertoir, I will accept the award. :roll:
Back to top
View user's profile Send private message
John R. Graham
Administrator
Administrator


Joined: 08 Mar 2005
Posts: 10261
Location: Somewhere over Atlanta, Georgia

PostPosted: Sat Feb 15, 2014 10:50 pm    Post subject: Reply with quote

I love to play the "I can do it better" game because about half the time someone else shows me how to do it better and I always learn something. Thank you for showing my how to retrieve that useful information from the package database. :)

- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters.
Back to top
View user's profile Send private message
dambacher
Apprentice
Apprentice


Joined: 11 Feb 2003
Posts: 288
Location: Germany

PostPosted: Sat Feb 15, 2014 11:15 pm    Post subject: Reply with quote

ok last words: how to find the cruft in etc wich does not belong to some installed package

Code:
find /var/db/pkg/ -name "CONTENTS" | xargs gawk '/obj\s\/etc/ { print $2 }'|sort > /tmp/f1
find /etc/ -type f |sort > /tmp/f2
diff -U 0 /tmp/f1  /tmp/f2|less


but not everything is cruft actually...

Code:
rm /tmp/f1 /tmp/f2
Back to top
View user's profile Send private message
gotyaoi
Tux's lil' helper
Tux's lil' helper


Joined: 01 Apr 2013
Posts: 137

PostPosted: Sun Feb 16, 2014 12:34 am    Post subject: Reply with quote

John R. Graham wrote:
khayyam wrote:
dambacher wrote:
[...] with my implementation find fires up cat 1126 times to gather data to stdout and then awk once to filter and format. with your implementation find fires up awk 1126 times directly, each time awk has to process it's command line to set up its filtering

dambacher ... your absolutely correct, that would make a difference. I was completely overlooking the numbers of files involved ...
The best answer is to launch neither one 1126 (or however many) times. This is exactly what xargs (part of sys-apps/findutils, like find) is for:
Code:
find /var/db/pkg/ -name "CONTENTS" | xargs gawk '/obj\s\/etc/ { print $3,$2}' | md5sum -c | grep -v "OK" | sort
This version launches cat zero times and gawk once (or a very few times) and is faster still. (I think khay's UUoC award stands.)

Happy to see you as well, khay. :)

- John


In the interests of pointlessness, I feel the need to point out that khay's version could be improved to match your with one simple change. Instead of
Code:
find /var/db/pkg/ -name "CONTENTS" -exec awk '/^obj[ ]\/etc/{print $3,$2}' {} \; | md5sum -c | grep -v "OK" | sort | less

just subsitute \; with +, like so.
Code:
find /var/db/pkg/ -name "CONTENTS" -exec awk '/^obj[ ]\/etc/{print $3,$2}' {} + | md5sum -c | grep -v "OK" | sort | less

That way, instead of calling gawk multiple times, it will be called once or very few times, with all the file names stuck end on end and passed to gawk that way.
Code:
-exec command {} +
       This  variant  of  the -exec action runs the specified command on the selected files, but the command line is built by appending each selected file name at the end; the total number of invocations of
       the command will be much less than the number of matched files.  The command line is built in much the same way that xargs builds its command lines.  Only one instance of `{}' is allowed  within  the
       command.  The command is executed in the starting directory.

It even mentions xargs there :)
Back to top
View user's profile Send private message
dambacher
Apprentice
Apprentice


Joined: 11 Feb 2003
Posts: 288
Location: Germany

PostPosted: Sun Feb 16, 2014 8:49 am    Post subject: Reply with quote

gotyaoi: mavelous
you show us what we should have learned years ago: RTFM! :D
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum