Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Tip/Trick: using tmpfs for /tmp
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
Cazzantonio
Bodhisattva
Bodhisattva


Joined: 20 Mar 2004
Posts: 4486
Location: Somewere around the world

PostPosted: Fri Dec 24, 2004 1:34 pm    Post subject: Reply with quote

with 1gb ram and several gigabytes of hd space to use as swap memory do you think it' safe to use
Code:
PORTAGE_TMPDIR=/tmp
?
Did someone have some problems using tmpfs to compile programs? Consider that a some of ram is used in compiling, if ram gets filled by tmpfs the system could get much slower...
I wonder if there is an option lo limit the amount of phisical ram used without limiting the usage of swap space
_________________
Any man's death diminishes me, because I am involved in mankind, and therefore never send to know for whom the bell tolls; it tolls for thee
-John Donne
Back to top
View user's profile Send private message
Gherald
Veteran
Veteran


Joined: 23 Aug 2004
Posts: 1399
Location: CLUAConsole

PostPosted: Fri Dec 24, 2004 2:45 pm    Post subject: Reply with quote

but tmpfs on swap is slower than reiserfs on disk
Back to top
View user's profile Send private message
Jinidog
Guru
Guru


Joined: 26 Nov 2003
Posts: 593
Location: Berlin

PostPosted: Fri Dec 24, 2004 10:31 pm    Post subject: Reply with quote

Okay, I made an emerge -e world using tmpfs for portage.
Nearly every package compiled around 30% faster than they compiled in earlier runs. (confirmed by genlop)
It's not 100% sure, because normaly compiling wasn't the only task of my PC (I surfed besides that) but this time it was.

I have 512 MB RAM and I closed everything during compiling and just had KDE up running.
_________________
Just unused Microsoft-Software is good Microsoft-Software
Back to top
View user's profile Send private message
drphibes
Guru
Guru


Joined: 30 Nov 2004
Posts: 432

PostPosted: Sat Dec 25, 2004 4:26 am    Post subject: Reply with quote

how much space do you need to emerge a big package like xorg or kde? i assume one would need to limit tmpfs sizes at mount time in order to guarantee you don't blow out your virtual memory....e.g. 512M max for a 1G physical memory box with, say a 1G swap. Is 512M enough space to emerge the big packages? autoclean must be "yes" right or your hosed?

doc
Back to top
View user's profile Send private message
hds
Advocate
Advocate


Joined: 21 Aug 2004
Posts: 2629
Location: Sprockhoevel [GER]

PostPosted: Sat Dec 25, 2004 4:34 am    Post subject: Reply with quote

Cazzantonio wrote:
with 1gb ram and several gigabytes of hd space to use as swap memory do you think it' safe to use
Code:
PORTAGE_TMPDIR=/tmp
?

emerge UT2004 :lol:

btw.. if you are using VMWARE, you cant do that either.
Back to top
View user's profile Send private message
ahorn
Guru
Guru


Joined: 01 Nov 2004
Posts: 366
Location: Ol' europe

PostPosted: Fri Jan 21, 2005 10:53 pm    Post subject: Reply with quote

if /tmp and /var/tmp is at tmpfs, it is cleaned out at boot. but gentoo cleans it at the booting process anyway.
how can i disable this to get some more speed at boot?
modify /etc/init.d/bootmisc, yes?

look here in bootmisc:
Code:
                #
                # Clean up /tmp directory
                #
                if [ -z "${CDBOOT}" ]
                then
                        ebegin "Cleaning /tmp directory"
                        #rm -f /tmp/.X*-lock /tmp/esrv* /tmp/kio* /tmp/jpsock.* /tmp/.fam* &>/dev/null
                        #rm -rf /tmp/.esd* /tmp/orbit-* /tmp/ssh-* /tmp/ksocket-* /tmp/.*-unix &>/dev/null
                        # Make sure our X11 stuff have the correct permissions
                        mkdir -p /tmp/.{ICE,X11}-unix &>/dev/null
                        chown root.root /tmp/.{ICE,X11}-unix &>/dev/null
                        chmod 1777 /tmp/.{ICE,X11}-unix &>/dev/null
                        eend 0
                fi

as you see
Code:
chmod 1777 /tmp/.{ICE,X11}-unix &>/dev/null

it is not necessary to mount tmpfs /tmp with mode=1777
_________________
Too many connections. Please try again later.
Back to top
View user's profile Send private message
ba747heavy
Apprentice
Apprentice


Joined: 11 Aug 2003
Posts: 216
Location: New Mexico, USA

PostPosted: Fri Jan 21, 2005 11:23 pm    Post subject: Reply with quote

Hmm, my computer likes to lock up, and it happens with any combination of programs running and not running. Now I wonder if the fact that my /tmp is a tmpfs has anything to do with it. Any way to know? I guess I can mount my /tmp as a normal partition and see what happens.
_________________
Fred Clausen
"leet [speak] is a cry for help from a shallow mind" - Doomwookie Jan 05
Back to top
View user's profile Send private message
beandog
Developer
Developer


Joined: 04 May 2003
Posts: 2063
Location: /usa/utah

PostPosted: Fri Jan 21, 2005 11:32 pm    Post subject: Reply with quote

Jinidog wrote:
strange.
On a system with 512 MB of RAM I tested the performance of being /var/tmp/portage a tmpfs.
The results:

without tmpfs:
real 2m41.081s
user 1m48.505s
sys 0m20.697s

with tmpfs:
real 2m58.270s
user 1m50.086s
sys 0m21.272s

Concluding the filesystem in the memory gave worse performance than having a normal Reiser3.6 Filesystem on the harddisk.
CCache was not activated.

I said it, strange.


I got the same result. Oh well.

Still, its cool to use as a replacement for /tmp at least.

Steve
_________________
If it ain't broke, tweak it. packages | dvds | blurays | blog | wiki
Back to top
View user's profile Send private message
MighMoS
Guru
Guru


Joined: 24 Apr 2003
Posts: 416
Location: @ ~

PostPosted: Sat Jan 22, 2005 3:20 am    Post subject: Reply with quote

I put /var/run in a tmpfs file system, so that should I lose power (we had power work being done for a while) or the machine freeze b/c of buggy NVidia drivers, programs wont get confused when they see their old PID files.
_________________
jabber: MighMoS@jabber.org

localhost # export HOME=`which heart`
Back to top
View user's profile Send private message
Master One
l33t
l33t


Joined: 25 Aug 2003
Posts: 754
Location: Austria

PostPosted: Mon Feb 14, 2005 7:05 pm    Post subject: Reply with quote

After reading this very interesting thread, I just wondered, how you guys would use tmpfs if you had a new IBM ThinkPad T42p with 2 GB RAM and a 4 GB swap partition. I'd like to setup my upcoming machine in the most efficient way. BTW Any info, if tmpfs operates well with hibernation of swsuspend2?
_________________
Las torturas mentales de la CIA
Back to top
View user's profile Send private message
Jinidog
Guru
Guru


Joined: 26 Nov 2003
Posts: 593
Location: Berlin

PostPosted: Mon Feb 14, 2005 8:28 pm    Post subject: Reply with quote

The only way is to benchmark, what is faster for you.

Concluding, I see no sence in mounting a temporary filesystem for the portage-working directory.
If there is enough free RAM, everything what compiling needs is cached in the RAM. It's the same like a filesystem in the RAM.

So, if there is not enough RAM, the cached files are removed from memory.
If it is a temporary filesystem and the memory is full, it has to be swapped, what needs time.
_________________
Just unused Microsoft-Software is good Microsoft-Software
Back to top
View user's profile Send private message
hds
Advocate
Advocate


Joined: 21 Aug 2004
Posts: 2629
Location: Sprockhoevel [GER]

PostPosted: Mon Feb 14, 2005 11:02 pm    Post subject: Reply with quote

Master One wrote:
4 GB swap partition

whatever came to your mind to setup a 4GB swap? false dope?
Back to top
View user's profile Send private message
Master One
l33t
l33t


Joined: 25 Aug 2003
Posts: 754
Location: Austria

PostPosted: Tue Feb 15, 2005 9:00 am    Post subject: Reply with quote

hds wrote:
whatever came to your mind to setup a 4GB swap? false dope?

You are right, that would indeed be a little overkill, was sticking with the general rule "twice the RAM"...

As I want to use hibernation on that machine, swap should be at least 2048 MB (or larger? didn't find the time for much reading on this subject with swsusp2 & hibernation).
_________________
Las torturas mentales de la CIA
Back to top
View user's profile Send private message
Erlend
Guru
Guru


Joined: 26 Dec 2004
Posts: 493

PostPosted: Thu Feb 17, 2005 2:54 pm    Post subject: Reply with quote

I've been curious about this for a while.

I notice that I have a gig or so in /tmp - it never seems to get automatically deleted. I'm not sure if I can delete all of it though.

I would guess that if people are having slower compile in tmpfs it's because they don't have enough available ram. Listen to the drive and see if it is using the swap partition?

I was wondering about putting and ext2 filesystem on /tmp /var/tmp and /boot, since this has very little overhead, nice and fast and it doesn't matter if any of this is corrupted.

Any ideas?

Thanks,

Erlend
Back to top
View user's profile Send private message
hds
Advocate
Advocate


Joined: 21 Aug 2004
Posts: 2629
Location: Sprockhoevel [GER]

PostPosted: Thu Feb 17, 2005 5:43 pm    Post subject: Reply with quote

Erlend wrote:

notice that I have a gig or so in /tmp - it never seems to get automatically deleted. I'm not sure if I can delete all of it though.

init 1
delete contents of /tmp
init 3 (or whatever your usual runlevel is) - voila

btw.. using ext2 for /boot and / over here, and ext3 for /data (just because i need largefile support [>4GB], otherwise i would use ext2 there as well).

ext2 is still the fastest, while takeing least CPU, IMHO.
and, if you ever f*cked up a reiserfs - sheesh, you will never use reiser again.
if you get hardware errors on your hdd (and this will hapen sooner or later to anyone of us), bet you run ext2 or ext3, because you will no be able to restore anything from your reiser.
beware of reiser, if your S.M.A.R.T will get full!! and until you realize hat, it is too late aleady ;(
Back to top
View user's profile Send private message
Erlend
Guru
Guru


Joined: 26 Dec 2004
Posts: 493

PostPosted: Thu Feb 17, 2005 6:23 pm    Post subject: Reply with quote

Quote:
init 1
delete contents of /tmp
init 3 (or whatever your usual runlevel is) - voila

Thanks for that.

Quote:
you will no be able to restore anything from your reiser.

Is that just because it is too complicated, and no recovery tools exist?

Quote:
beware of reiser, if your S.M.A.R.T will get full!! and until you realize hat, it is too late aleady ;(

Not sure what you mean I'm afraid?

Erlend
Back to top
View user's profile Send private message
hds
Advocate
Advocate


Joined: 21 Aug 2004
Posts: 2629
Location: Sprockhoevel [GER]

PostPosted: Thu Feb 17, 2005 6:53 pm    Post subject: Reply with quote

Erlend wrote:

hds wrote:
you will no be able to restore anything from your reiser.

Is that just because it is too complicated, and no recovery tools exist?

no, recovery tools do exist, but infact of a hardware defect of the HDD, you are simply lost. the recovery tools will fail, at least they did here (reiser3, that was).
they will try to move data, or write a new superblock (depends on the options you give), and fail, because the next block they try will get unwritable as well. this gets into a ratrace, marking blocks as bad all and all over ;(

hds wrote:
beware of reiser, if your S.M.A.R.T will get full!! and until you realize hat, it is too late aleady ;(

Erlend wrote:

Not sure what you mean I'm afraid?

i am talking about the harddrives internal "bad block table". if that gets full, an IDE drive will get more and more corrupted with bad blocks at any write acess you do.
in this case, one prefers to save as much data as possible, but reiser will also deny readaccess in this case, because its journal is corrupted!

if you come from DOS, you might be familiar with "lost+found" directory, thats what "checkdisk" did in the very old days.
btw: this is not against reiser in general, i guess this will happen with any journaling filesys infact of hardware defects.

and btw: yes, i do backups, but you are always caught if you did something you wouldnt like to loose. shit happens :evil:
Back to top
View user's profile Send private message
Erlend
Guru
Guru


Joined: 26 Dec 2004
Posts: 493

PostPosted: Thu Feb 17, 2005 10:22 pm    Post subject: Reply with quote

Thanks. I didn't know any of that.

I guess one advantage of having /tmp /var/tmp in RAM is that it will increase the lifetime of your hard disk - writing/reading small files like crazy can't be particularly good for it. On that note, is "emerge -e world" "bad" for your CPU? I'm just wondering if making your computer compile for 30 odd hours is going to affect it's performance/life?

Also, could one not use device-mapper:
echo "0 X linear /dev/ram0 0 /dev/hda4 0" | dmsetup create tmpHybrid
...
mount /dev/mapper/tmpHybrid
So that the first 50MB of /tmp is RAM, and the rest ext2 space?

Erlend
Back to top
View user's profile Send private message
Master One
l33t
l33t


Joined: 25 Aug 2003
Posts: 754
Location: Austria

PostPosted: Fri Feb 18, 2005 12:17 pm    Post subject: Reply with quote

MighMoS wrote:
I put /var/run in a tmpfs file system, so that should I lose power (we had power work being done for a while) or the machine freeze b/c of buggy NVidia drivers, programs wont get confused when they see their old PID files.

This makes no sense at all, because /var/run get's cleaned on every boot anyway, see /etc/init.d/bootmisc:

Code:
             ebegin "Cleaning /var/lock, /var/run"
                rm -rf /var/run/console.lock /var/run/console/*
                if [[ -z ${CDBOOT} ]] ; then
                        #
                        # Clean up any stale locks.
                        #
                        find /var/lock -type f -print0 | xargs -0 rm -f --
                        #
                        # Clean up /var/run and create /var/run/utmp so that we can login.
                        #
                        for x in $(find /var/run/ ! -type d ! -name utmp ! -name innd.pid ! -n$
                                local daemon=${x##*/}
                                daemon=${daemon%*.pid}
                                # Do not remove pidfiles of already running daemons
                                if [[ -z $(ps --no-heading -C "${daemon}") ]] ; then
                                        if [[ -f ${x} || -L ${x} ]] ; then
                                                rm -f "${x}"
                                        fi
                                fi
                        done
                fi

_________________
Las torturas mentales de la CIA
Back to top
View user's profile Send private message
markandrew
Apprentice
Apprentice


Joined: 04 Feb 2004
Posts: 176
Location: Manchester, UK

PostPosted: Fri Feb 18, 2005 3:00 pm    Post subject: Reply with quote

I had /tmp and /var/tmp as tmpfs for a while but went back to normal fs (reiser) after a while as i saw no noticeable speed increase in general usage or during compiling, and emerging large packages, like xorg or openoffice, would fill out the /var/tmp dir and i'd have to start over. dedicating more memory, i decided, wasn't worth it (got 1G ram).

I also had, for a while, my /var/logs dir as tmpfs, with it being read from disk on boot and written back on shutdown. however after a couple of crashes (non related) i realized that if the machine didn't shutdown properly, i lost all my logs for that session... and thus couldn't figure out why the crashes were happening.

in the end i decided it was a nice idea, but didn't give any real benefits.
Back to top
View user's profile Send private message
Erlend
Guru
Guru


Joined: 26 Dec 2004
Posts: 493

PostPosted: Fri Feb 18, 2005 8:11 pm    Post subject: Reply with quote

I've always wondered it having /etc/init.d in ram at boot would speed things up. I imagine it wouldn't make much difference though?

Just a though, /var is temporary in that it is meant for files which are non-essential, and which don't change very often: surely then that could be tmpfs or some low-overhead filesystem (ext2).

Erlend
Back to top
View user's profile Send private message
kallamej
Administrator
Administrator


Joined: 27 Jun 2003
Posts: 4919
Location: Gothenburg, Sweden

PostPosted: Fri Mar 25, 2005 8:07 am    Post subject: Reply with quote

Erlend wrote:
Just a though, /var is temporary in that it is meant for files which are non-essential, and which don't change very often: surely then that could be tmpfs or some low-overhead filesystem (ext2).
Do not use tmpfs for /var, portage and other programs depend on it surviving a reboot.
_________________
Please read our FAQ Forum, it answers many of your questions.
irc: #gentoo-forums on irc.freenode.net
Back to top
View user's profile Send private message
lost+found
Guru
Guru


Joined: 15 Nov 2004
Posts: 508
Location: North~Sea~Coa~s~~t~~~

PostPosted: Tue Apr 19, 2005 1:10 pm    Post subject: Reply with quote

This is how I made a virtual Test/Guest Account in tmpfs /tmp.

Code:
# useradd test -m -G users,audio,tty -s /bin/bash -d /tmp/test
# passwd test


/etc/conf.d/local.start:
Code:
cp -R /etc/skel /tmp
rename skel test /tmp/skel
chmod 0700 /tmp/test
chown -R test:users /tmp/test


Restart...
Back to top
View user's profile Send private message
mr.ed
Tux's lil' helper
Tux's lil' helper


Joined: 30 Nov 2004
Posts: 75

PostPosted: Sun Apr 24, 2005 1:56 pm    Post subject: Reply with quote

K, here r my two cents...

Quote:
[15:29][root:-> /]@: df
/dev/sda3 45G 5.8G 39G 14% /
udev 1005M 600K 1004M 1% /dev
tmpfs 1.0G 1.4M 1023M 1% /dev/shm
/dev/sdb1 66G 24G 43G 36% /home


Quote:
0 lrwxrwxrwx 1 root root 12 Apr 12 15:25 /tmp -> /dev/shm/tmp
0 lrwxrwxrwx 1 root root 12 Apr 12 15:25 /var/tmp -> /dev/shm/tmp


This is my fstab-line for /dev/shm (nothing else!)
Quote:
tmpfs /dev/shm tmpfs defaults,size=1G 0 0


As u can see my tmp-dir is mounted at /dev/shm (which is the standard tmpfs). I noticed this thread before and thought if it would be possible to redirect all of your tmp-dirs to /dev/shm and thus move all of your workfiles to tmpfs... After a bit of searching round i came to the conclusion it can be done and it's a real speedgain to. At first i tried the fstab solution, but it gave 2 much trouble (mount bind and stuff). But then i realised a symlink would do exact the same thing!!!

Basically i just deleted all of the tmp-dirs first and then created the dir /dev/shm/tmp. Then I chmod 1777 /dev/shm/tmp (NOT /dev/shm) and symlinked the tmp-dirs to this folder.
The only downside is off course u gotta have enough ram to compile larger progs like xorg, kde (for OOo or anything else larger then my tmpfs i remount /var/tmp/portage to a partition). But then u would still have the problem of your ram filling up rather quickly and not getting cleaned... My solution was emerging tmpwatch and modifying the cronscript in such a way that it cleans out my /var/tmp/portage every hour of files that haven't been accessed for one hour and cleans out /tmp every 8 hours. I added the script to cron to be run every half hour, so the cache never outgrows my tmp!!!
The second problem was that the /dev/shm/tmp get's deleted off course at every reboot, so i added this to /etc/init.d/localmount:
Quote:
start() {
# Make tmp directories for mounting /etc/fstab.
mount /dev/shm
mkdir /dev/shm/tmp
chmod 1777 /dev/shm/tmp

# Mount local filesystems in /etc/fstab.
ebegin "Mounting local filesystems"


Any1 paying attention will see that this combination creates a tmp-dir IN tmpfs, not a dir mounted on tmpfs (cause it allready exists with /dev/shm, see your own df -h)
If some1 has questions bout it, i'ld be willing to set up some sort of guide to how to do it...

P.s.: I got 2 gigs of ddr2 ram and i can leave my comp on for days in a row if i'ld like, it runs the same as if temp was normally mounted at a filesystem, but faster ;-)... This setup i've been using for 3 weeks now and never got into trouble with tmp filling up!! (Never grew to more then 75%)
_________________
(Ex-Pornstar)


Last edited by mr.ed on Sun Apr 24, 2005 9:24 pm; edited 1 time in total
Back to top
View user's profile Send private message
lost+found
Guru
Guru


Joined: 15 Nov 2004
Posts: 508
Location: North~Sea~Coa~s~~t~~~

PostPosted: Sun Apr 24, 2005 6:13 pm    Post subject: Reply with quote

mr.ed wrote:
...but it gave 2 much trouble (mount bind and stuff)...


The mount --bind thing is easy... :-)

This makes /dev/shm and /tmp share one tmpfs.

/etc/conf.d/local.start:
Code:
cp -Rp /tmp /dev/shm
mount --bind /dev/shm/tmp /tmp
:!: cp instead of mkdir, this avoids X11 crash.

/etc/fstab:
Code:
none  /dev/shm  tmpfs  nodev,nosuid,noexec,noatime,size=256M  0 0


Restart...
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page Previous  1, 2, 3  Next
Page 2 of 3

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum