Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
How to minimize writes to SSD?
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
Jt0
n00b
n00b


Joined: 28 Oct 2018
Posts: 32

PostPosted: Tue Dec 18, 2018 3:12 pm    Post subject: How to minimize writes to SSD? Reply with quote

In setting up my partition scheme, I'd like to avoid excessive read/write cycles on my SSD. As I understand it, compilation requires a lot of these, so wherever packages (ebuilds?) are built should be mounted to an HDD.

Another possibility would be to have the directory/directories mounted as ramdisks, although I don't know what kind of setup is required there or if it's even a feasible partitioning scheme.

What's the best way to transfer the load of compilation off of an SSD?
Back to top
View user's profile Send private message
fedeliallalinea
Bodhisattva
Bodhisattva


Joined: 08 Mar 2003
Posts: 22174
Location: here

PostPosted: Tue Dec 18, 2018 3:16 pm    Post subject: Reply with quote

https://wiki.gentoo.org/wiki/Portage_TMPDIR_on_tmpfs
_________________
Questions are guaranteed in life; Answers aren't.
Back to top
View user's profile Send private message
Jt0
n00b
n00b


Joined: 28 Oct 2018
Posts: 32

PostPosted: Tue Dec 18, 2018 3:43 pm    Post subject: Reply with quote

Thanks, this was exactly what I was looking for!

On a related note, will this require me to use an initramfs? From the handbook:
Quote:
In certain cases it is necessary to build an initramfs - an initial ram-based file system. The most common reason is when important file system locations (like /usr/ or /var/) are on separate partitions. With an initramfs, these partitions can be mounted using the tools available inside the initramfs.

My understanding from this is that if your system is partitioned such that everything required for booting is not on one single partition, you have to specify in an initramfs to mount all of these partitions. I wouldn't think this would include the Portage tmp files, and the wiki page doesn't mention anything about the initramfs. Is that right?
Back to top
View user's profile Send private message
Jaglover
Watchman
Watchman


Joined: 29 May 2005
Posts: 7196
Location: Saint Amant, Acadiana

PostPosted: Tue Dec 18, 2018 4:00 pm    Post subject: Reply with quote

Portage does need this space when the computer is booting. So the answer is no.
_________________
Please learn how to denote units correctly!
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43557
Location: 56N 3W

PostPosted: Tue Dec 18, 2018 4:36 pm    Post subject: Reply with quote

Jt0,

You only need an initrd if the kernel cannot boot unaided. Like, you have root in lvm or raid or encrypted.
Then you need user space tools before root can be seen by the kernel.

There is one other case. Due to what I consider poor design decisions by ${UPSTREAM} /usr must be mounted before udev starts.
You also need an initrd if you have a separate /usr partition and use the normal udev or its replacements.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
papu
l33t
l33t


Joined: 25 Jan 2008
Posts: 605
Location: Sota algun pi o alzina...

PostPosted: Tue Dec 18, 2018 5:26 pm    Post subject: Reply with quote

i use tmpfs for compile on ram and plus a daily cron for trim, like this

Code:

fstab
tmpfs                     /tmp         tmpfs      size=10G,noatime,nodev,nosuid            0   0

make.conf
PORTAGE_TMPDIR="/tmp"

i have two partions:
~ $ cat /etc/cron.daily/trim
#!/bin/sh

LOG=/var/log/trim.log
echo "*** $(date -R) ***" >> $LOG
fstrim -v / >> $LOG
fstrim -v /boot >> $LOG

_________________
--Intel i5 6600 --GA-Z170-HD3P --Ripjaws V 2x8 2400 --Radeon RX 470 G1 4G --Gentoo & Windows
Back to top
View user's profile Send private message
depontius
Advocate
Advocate


Joined: 05 May 2004
Posts: 3387

PostPosted: Wed Dec 19, 2018 6:58 pm    Post subject: Reply with quote

My /var/tmp/portage has been sitting on tmpfs for some time now, even though I have a spinning hard drive, just to cut compilation times.

I've spoken to Santa-wife about an SSD later this month, so in preparation I've been working on a "kernelPrep" script that creates an overlayfs setup. The kernel source proper is on the drive, the overlay is on tmpfs.
_________________
.sigs waste space and bandwidth
Back to top
View user's profile Send private message
Anon-E-moose
Advocate
Advocate


Joined: 23 May 2008
Posts: 4207
Location: Dallas area

PostPosted: Wed Dec 19, 2018 8:08 pm    Post subject: Reply with quote

Ram is cheap enough, in most cases.

I bumped my system to 32gig a few years ago and have both /tmp and /var/tmp/portage as tmpfs.
I set them up to automount from /etc/fstab and don't worry about it.

Main disk is samsung evo ssd with /boot, /, /usr/portage and /x (storage) partitions even with writes to /var/log, /usr/portage, /usr/src and /home, I shouldn't have a problem for a long time as far as write lifetime goes.

I've got an older 250gb 850 evo and it's rated at 75TBwrites (conservative estimate)
But I bought the 860 evo models earlier this year (for future replacements) and they double the write estimate of the 850s. (500gb has 300tbw and 1tb has 600tbw)

And the price on samsung evo's has dropped a lot this year, with 250gb running ~60-70, 500gb running ~80-120 and 1tb's for as low as 150 on sale.

Edit to add: did a rough estimate and I've written 2.7TB in 4 years so given that, I should have a life expectancy of over 100 years at my current rate.
Note, I update the /usr/portage partition daily and constant writes to /var/log and /home/<browser directories>
_________________
Asus m5a99fx, FX 8320 - nouveau, oss4, rx550 for qemu passthrough
Acer laptop E5-575, i3-7100u - i965, alsa
---both---
5.0.13 zen kernel, profile 17.1 (no-pie & modified) amd64-no-multilib
gcc 8.2.0, eudev, openrc, openbox, palemoon


Last edited by Anon-E-moose on Wed Dec 19, 2018 8:51 pm; edited 2 times in total
Back to top
View user's profile Send private message
joanandk
Tux's lil' helper
Tux's lil' helper


Joined: 12 Feb 2017
Posts: 87

PostPosted: Wed Dec 19, 2018 8:37 pm    Post subject: Re: How to minimize writes to SSD? Reply with quote

Jt0 wrote:
I'd like to avoid excessive read/write cycles on my SSD. As I understand it, compilation requires a lot of these,


I have been using this SSD for three years now. This is the output of the script from Ubuntu:
Code:

------------------------------
 SSD Status:   /dev/sda
------------------------------
 On time:      2,731 hr
------------------------------
 Data written:
           MB: 2,025,210.739
           GB: 1,977.744
           TB: 1.931
------------------------------
 Mean write rate:
        MB/hr: 741.563
------------------------------
 Drive health: 99 %
------------------------------


This disk holds the entire Gentoo, 4 VMs, one is heavily used. The VM contains a gentoo installation too. As the drive is a Samsung 850 Pro, 450TBW is the life expectancy.

If I calculate, I can keep my pace as it is now and it would take around 600 years for the device to get to 450TB. I would say, if you are not updating your system every single day, but on average 2x a month, you have less than anything to worry about!

BR
Back to top
View user's profile Send private message
Dwosky
Tux's lil' helper
Tux's lil' helper


Joined: 07 Nov 2018
Posts: 87

PostPosted: Thu Dec 20, 2018 7:39 am    Post subject: Reply with quote

I saw some time ago an analysis over SSD writing capacity and I believe the article end writing until the SSD started to fail, then accounted the total size written and the average was something like 10-20Gb if I recall correctly.

Its true that SSD have a limited spawn life, but the actual and previous units are way better than the initial ones, where the issue was notable. I've checked another article: https://techreport.com/review/27909/the-ssd-endurance-experiment-theyre-all-dead in which you can see the units start failing after writing thousands of terabytes, which its a huge amount for the average user.

In summary its ok to move the tasks that write the most, such as portage tmpfs, but in the end there is no really need to worry in excess beside that. In my case I have a server with an SSD over 55k active hours and its still working without issues (I have only portage tmp in RAM with tmpfs).
Back to top
View user's profile Send private message
krumpf
n00b
n00b


Joined: 15 Jul 2018
Posts: 15

PostPosted: Thu Dec 20, 2018 8:26 am    Post subject: Reply with quote

My 2 cents tip: to avoid using my SSD, I moved the Firefox cache folder to a HDD
In about:config url, create a new string value called browser.cache.disk.parent_directory and set the path to wherever it pleases you
You can then check with about:cache url
(It might be possible with others browsers, but i don't know the how-to)
Back to top
View user's profile Send private message
Keruskerfuerst
Advocate
Advocate


Joined: 01 Feb 2006
Posts: 2288
Location: near Augsburg, Germany

PostPosted: Thu Dec 20, 2018 8:50 am    Post subject: Reply with quote

Today, the live span (data written to SSD) is equal to to normal HDD.

I have read an article, that the write volume of a Samsung 850 Pro SSD is 2 PB.
Back to top
View user's profile Send private message
pietinger
n00b
n00b


Joined: 17 Oct 2006
Posts: 55

PostPosted: Thu Dec 20, 2018 9:24 am    Post subject: Re: How to minimize writes to SSD? Reply with quote

Jt0 wrote:
In setting up my partition scheme, I'd like to avoid excessive read/write cycles on my SSD. As I understand it, compilation requires a lot of these, so wherever packages (ebuilds?) are built should be mounted to an HDD.

I spent /usr/portage an extra partition on my HDD (and of course doing the compilation in RAM). Here is my fstab (sda is the SSD, sdb is the HDD):
Code:

/dev/sda2       /boot                   vfat    defaults,noauto,noatime                 1 2
/dev/sda3       /                       ext4    defaults,noatime                        0 1
/dev/sdb1       none                    swap    sw                                      0 0
/dev/sdb2       /usr/portage            ext4    defaults,noatime                        0 2
/dev/sdb3       /hd                     ext4    defaults,noatime                        0 2
tmpfs           /tmp                    tmpfs   rw,noexec,nodev,nosuid,mode=1777        0 0
tmpfs           /var/tmp/portage        tmpfs   rw,nodev,nosuid,size=10G,uid=portage    0 0
/dev/cdrom      /mnt/cdrom              auto    noauto,user                             0 0
/dev/sdc1       /mnt/hdu                auto    noauto,user,noatime                     0 0
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43557
Location: 56N 3W

PostPosted: Thu Dec 20, 2018 5:50 pm    Post subject: Reply with quote

pietinger,

/usr/portage itself, being a lot of small files, benefits from SSD.
/usr/portage/distfiles and /usr/portage/packages, which are large sequentially accessed files can be on rotating rust an that's something that rotating rust is good at. Its still slower than SSD but there is almost no random access, so it won't matter.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
pietinger
n00b
n00b


Joined: 17 Oct 2006
Posts: 55

PostPosted: Fri Dec 21, 2018 2:36 pm    Post subject: Reply with quote

Hi Neddy,

thank you for your answer. You are right with

NeddySeagoon wrote:
/usr/portage itself, being a lot of small files, benefits from SSD.

But I want to save my (old and not so modern) SSD and dont care about the time my emerge needs, because emerge runs only every night. The second reason is the size of /usr/portage (at the moment 19 GB). If you have a small and old SSD and your root-partition (with home) is only 15 GB I find it useful to put all files which are not time-sensitiv on the 2 TB HDD. Before many years (10 or more) there was also a recommandation for an extra partition for /usr/portage in the installation manual.
(My swap-partition is also on the hdd, because I have no swapping).
Back to top
View user's profile Send private message
Chiitoo
Administrator
Administrator


Joined: 28 Feb 2010
Posts: 1803
Location: Here and Away Again

PostPosted: Fri Dec 21, 2018 2:44 pm    Post subject: ><)))°€ Reply with quote

pietinger wrote:
The second reason is the size of /usr/portage (at the moment 19 GB).

I like to make use of 'DISTDIR' and 'PKGDIR', with which my '/usr/portage' is only around 700 MiB right now. :]
_________________
Kind Regards,
~ The Noob Unlimited ~

Sore wa sore, kore wa kore.
Back to top
View user's profile Send private message
Anon-E-moose
Advocate
Advocate


Joined: 23 May 2008
Posts: 4207
Location: Dallas area

PostPosted: Fri Dec 21, 2018 2:46 pm    Post subject: Reply with quote

pietinger wrote:
The second reason is the size of /usr/portage (at the moment 19 GB).


Why so huge, mine is 4gb w/3.5g of it being distfiles.

You probably need to run eclean-dist, run it as "eclean-dist -p" to see what it would do, then if satisfied just "eclean-dist" (as root, of course)
_________________
Asus m5a99fx, FX 8320 - nouveau, oss4, rx550 for qemu passthrough
Acer laptop E5-575, i3-7100u - i965, alsa
---both---
5.0.13 zen kernel, profile 17.1 (no-pie & modified) amd64-no-multilib
gcc 8.2.0, eudev, openrc, openbox, palemoon
Back to top
View user's profile Send private message
pietinger
n00b
n00b


Joined: 17 Oct 2006
Posts: 55

PostPosted: Fri Dec 21, 2018 2:57 pm    Post subject: Reply with quote

Anon-E-moose wrote:
pietinger wrote:
The second reason is the size of /usr/portage (at the moment 19 GB).


Why so huge, mine is 4gb w/3.5g of it being distfiles.

You probably need to run eclean-dist, run it as "eclean-dist -p" to see what it would do, then if satisfied just "eclean-dist" (as root, of course)


Thanks - I know. I dont delete old distfiles, because I want to save all the old version in the case I need one (was only needful one time since my initial installation in 2017). I will do an eclean-dist when my /usr/portage will reach 32 GB (the whole partition size is 64 GB; almost nothing on a 2 TB HDD).
Back to top
View user's profile Send private message
depontius
Advocate
Advocate


Joined: 05 May 2004
Posts: 3387

PostPosted: Fri Dec 21, 2018 3:29 pm    Post subject: Reply with quote

NeddySeagoon wrote:
pietinger,

/usr/portage itself, being a lot of small files, benefits from SSD.
/usr/portage/distfiles and /usr/portage/packages, which are large sequentially accessed files can be on rotating rust an that's something that rotating rust is good at. Its still slower than SSD but there is almost no random access, so it won't matter.


I nfs-mount both /usr/portage and /usr/portage/distfiles. For the former, I sync it every night on the server, and it's exported ro to the clients. The latter is exported rw to the clients, so that whoever wants a package first, gets it for everyone. Besides, client installations are not identical.
_________________
.sigs waste space and bandwidth
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43557
Location: 56N 3W

PostPosted: Fri Dec 21, 2018 5:08 pm    Post subject: Reply with quote

depontius,

I choose not to use nfs as my rsync server is in my DMZ. It doubles as a mail server too, so its not trusted.
I use http-replicator for distfiles so they are cached on first use.

My portage repro is 310MB but its on its own filesystem with 1k blocks. As I write that, I realise that the SSD underneath uses 4k blocks. Oops.
Hopefully the kernel does write combining.
distfiles, also on its own rotating rust filesystem is 131G but I never throw any away. They have been accumulating since April 2009.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Anon-E-moose
Advocate
Advocate


Joined: 23 May 2008
Posts: 4207
Location: Dallas area

PostPosted: Fri Dec 21, 2018 7:12 pm    Post subject: Reply with quote

Depends on the ssd, Neddy. My samsung uses 512k blocks and I'm on a btrfs file system.

tarred size is 412M, and du -sh reports 682M (portage minus distfiles)

Like you I've kept all my distfiles, I have a usb raid (mirrored) that I rsync distfiles to nightly along with tarring up the whole portage directory (minus distfiles) onto it.
I also rsync root nightly and keep a rolling 4 week root done on sunday morning. It's all on my internal network, so I let my laptop rsync to it for emerge syncing and picking up distfiles.
_________________
Asus m5a99fx, FX 8320 - nouveau, oss4, rx550 for qemu passthrough
Acer laptop E5-575, i3-7100u - i965, alsa
---both---
5.0.13 zen kernel, profile 17.1 (no-pie & modified) amd64-no-multilib
gcc 8.2.0, eudev, openrc, openbox, palemoon
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum