Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Speeding up locate
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
abysed
n00b
n00b


Joined: 06 Mar 2003
Posts: 39
Location: Seattle, Washington

PostPosted: Thu Aug 28, 2003 4:29 am    Post subject: Speeding up locate Reply with quote

I normally use locate -u to update the database, but that will be changing soon. I wanted to take some paths out of the database file, such as /usr/portage, since it would often give a lot of unnecessary output when trying to locate files. I did this on my laptop, a P4 2.2GHz machine with 256MB using vanilla-sources-2.4.22 and no swap partition. I also have both KDE and Gnome on here, so my database will be larger than most. My goal behind this was to speed up locates, as well as spend less time updating.

* The database refers to /var/lib/slocate/slocate.db
* Changes should be put in /etc/updatedb.conf

Note that using locate -u doesn't read /etc/updatedb.conf, thus the database size will be larger. Just from using updatedb, I gained 1MB back.

I did some tests along the way just to see my progress with file size (and to show you, of course).
Quote:
Using locate -u
database size: 4.02MB

Using updatedb with the default PRUNEPATHS="/tmp /var/tmp"
database size: 3.08MB

Using updatedb with PRUNEPATHS="/tmp /var/tmp /usr/portage"
database size: 2.28MB

Using updatedb with PRUNEPATHS="/tmp /var/tmp /usr/portage /usr/src /var/db"
database size: 1.45MB

So, I went from 4MB down to 1.45MB, I'm happy for now ;)

And now for some tests for time updating the database... I rebooted before doing each one to make sure paths weren't stored in memory.

Code:

james slocate # time locate -u
real 7m24.406s
user 0m0.950s
sys 0m11.010s

Using PRUNEPATHS="/tmp /var/tmp /usr/portage /usr/src /var/db"
Code:

james slocate # time updatedb
real 2m29.117s
user 0m0.320s
sys 0m1.850s

7m24s down to 2m29s! Pretty good I think :)

My servers database went from 1.48MB to 483KB, 66% smaller!

Shoutouts to opello (laptop went from 4.4MB to 2.108MB, server: 3.4MB to 788KB) and r2d2 (laptop: 4.0MB to 2.3MB), who gave me the idea of putting /usr/src and /var/db in PRUNEPATHS. Saved me another 800KB :)
Back to top
View user's profile Send private message
cleber
n00b
n00b


Joined: 05 Aug 2003
Posts: 74
Location: São Paulo - Brazil

PostPosted: Thu Aug 28, 2003 5:05 am    Post subject: Reply with quote

I am a adm on a RH machine running on a PC with a normal IDE HDD, and it makes updatedb within :
Code:

[root@node8 /home/cleber]# time updatedb

real    0m57.168s
user    0m0.190s
sys     0m0.820s


and, in the /etc/updatedb.conf:
Code:
PRUNEFS="devpts NFS nfs afs proc smbfs autofs auto iso9660"
PRUNEPATHS="/tmp /usr/tmp /var/tmp /afs /net"
export PRUNEFS
export PRUNEPATHS


This is incredibly fast...
_________________
Saru mo ki kara ochiru - Japanese proverb.
Cleber Mori Home Page: http://www.2ks.com.br
Back to top
View user's profile Send private message
rongten
Tux's lil' helper
Tux's lil' helper


Joined: 18 Apr 2003
Posts: 116
Location: N 50° 45.338' E 04° 23.139'

PostPosted: Fri Dec 10, 2004 8:07 am    Post subject: Reply with quote

vax,

thanks for pointing out this config file.

You never finish learning with Linux...

Bye
_________________
Nothing is ever easy
#1401 @ Associate FSF Membership
Join FSF as an Associate Member!(With Referrer)
What are you waiting for?
Back to top
View user's profile Send private message
-Rick-
Tux's lil' helper
Tux's lil' helper


Joined: 29 Aug 2004
Posts: 77
Location: Holland

PostPosted: Fri Dec 10, 2004 1:46 pm    Post subject: Reply with quote

Nice tip, always wanted to shorten the time of updatedb :)
_________________
Cube bots
Nixstaller - Easy creatable installers for *nix
Back to top
View user's profile Send private message
Douglas
n00b
n00b


Joined: 09 Dec 2004
Posts: 14
Location: Bunker Delta

PostPosted: Fri Dec 10, 2004 4:20 pm    Post subject: Reply with quote

Nice tip!

Another thing you can do is make updatedb a weekly cron job rather than a daily one. For me weekly is plenty good enough and I can always run updatedb if I need to.
Code:
mv /etc/cron.daily/slocate /etc/cron.weekly

Although if you update cron-base, you need to go into /etc and delete slocate from cron.daily again.
_________________
Calculating BogoMIPS... Integer Overflow!
Kernel Panic! System Halted
Back to top
View user's profile Send private message
rhill
Retired Dev
Retired Dev


Joined: 22 Oct 2004
Posts: 1629
Location: sk.ca

PostPosted: Fri Dec 10, 2004 10:05 pm    Post subject: Reply with quote

root ~ # time updatedb
real 0m1.016s
user 0m0.544s
sys 0m0.459s
root ~ #
root ~ # rm /var/lib/slocate/slocate.db
removed `/var/lib/slocate/slocate.db'
root ~ # time updatedb
warning: updatedb: could not open database: /var/lib/slocate/slocate.db: No such file or directory

real 0m1.067s
user 0m0.565s
sys 0m0.443s

might be a disk cache or something, but before avg time was 12-60 seconds.

Code:
# paths which are pruned from updatedb database
PRUNEPATHS="/dev /proc /media /mnt /sys /tmp /var/db /var/tmp /usr/portage /var/cache /usr/src"


/usr/src cut off 8 seconds, but i can think of times i might need it, so i took it out again. 8 seconds is fine.
Back to top
View user's profile Send private message
dkure
n00b
n00b


Joined: 20 Apr 2004
Posts: 51
Location: Sydney, Australia

PostPosted: Thu Dec 16, 2004 3:00 pm    Post subject: Reply with quote

dramatic drop in size for me.

Nearly halved.

Before 11MB
After 5.9MB

Takes only 2m 30s to build now aswell, an added bonus :D

Code:
PRUNEPATHS="/dev /proc /tmp /var/tmp /usr/portage /var/db /sys /usr/local/overlays"
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum