Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
too much diskspace used
View unanswered posts
View posts from last 24 hours

Goto page 1, 2, 3  Next  
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
josephg
l33t
l33t


Joined: 10 Jan 2016
Posts: 783
Location: usually offline

PostPosted: Fri Jan 15, 2016 4:11 am    Post subject: too much diskspace used Reply with quote

i am yet to finish my install, haven't yet started installing x. and i have already exceeded 4g diskspace.
i don't know whether this is expected. it could be due to some mistakes i made in selecting the wrong profile and use flags, which has been rectified (i think). i've compiled the whole world a couple of times now.

how can i reduce the clutter?

my main aim in moving to gentoo was to have and maintain a stable, lean & clean system. is there a guide or link someone could point me to?
Back to top
View user's profile Send private message
Keruskerfuerst
Advocate
Advocate


Joined: 01 Feb 2006
Posts: 2288
Location: near Augsburg, Germany

PostPosted: Fri Jan 15, 2016 6:17 am    Post subject: Reply with quote

For a normal install you need a free disk space of ~20GB.
Back to top
View user's profile Send private message
josephg
l33t
l33t


Joined: 10 Jan 2016
Posts: 783
Location: usually offline

PostPosted: Fri Jan 15, 2016 6:33 am    Post subject: Reply with quote

Keruskerfuerst wrote:
For a normal install you need a free disk space of ~20GB.

really?? i was hoping it won't be more than 2gb :roll: all my other installs (except ubuntu, and another one that shan't be mentioned) are at that level. and i still try to prune them down.

i guess everybody has a different idea of normal. my normal is barebones.. busybox, minimal x and a tiny wm.

my debian wheezy32 is less than 1gb, and that is with x, firefox, flash, sylpheed, mplayer, libreoffice, etc. i have everything i need on this. except that i don't see myself wanting to keep up with debian tactics.

i have 12 partitions on my laptop disk, each (except one) with a different distro ;)
Back to top
View user's profile Send private message
Irre
Guru
Guru


Joined: 09 Nov 2013
Posts: 340
Location: Stockholm

PostPosted: Fri Jan 15, 2016 7:51 am    Post subject: Reply with quote

My gentoo-systems need from 2Gb to about 6Gb!

But they all share some maps that are common and not necessary to backup, specified in file /etc/portage/make.conf:
Code:
DISTDIR="/sdb2/distfiles"
PORTDIR="/sdb2/portage"
PORTAGE_BINHOST="/sdb2/transfer/bindist64"
PKGDIR="/sdb2/transfer/bindist64"

and in /etc/fstab:
Code:
192.168.1.2:/ /sdb2 nfs rw,noatime,_netdev 0 0
(A common fileserver in my case)

Some systems also share: /var/tmp for heavy builds like gcc.

This way you can have several systems on the same harddisk and you can keep disk image backups without unimportant data!
Back to top
View user's profile Send private message
khayyam
Watchman
Watchman


Joined: 07 Jun 2012
Posts: 6228
Location: Room 101

PostPosted: Fri Jan 15, 2016 2:48 pm    Post subject: Reply with quote

Keruskerfuerst wrote:
For a normal install you need a free disk space of ~20GB.

Keruskerfuerst ... really?

Code:
# df -h
Filesystem           Size  Used Avail Use% Mounted on
/dev/mapper/vg-root  9.8G  3.6G  5.7G  39% /
mdev-tmpfs            10M   56K   10M   1% /dev
tmpfs                201M  156K  201M   1% /run
shm-tmpfs           1003M     0 1003M   0% /dev/shm
/dev/mapper/vg-var   5.8G  882M  4.7G  16% /var
/dev/mapper/vg-home  276G  228G   49G  83% /home

... so that is about 4.5G, and that includes all sources required for a complete (--empty-tree) rebuild, plus kernel sources, and an Aboriginal Linux chroot.

@josephg ... did you remove the stage3 and portage-snapshot subsequent to unpacking? You might also save some space by running 'make clean' on /usr/src/linux-{version} ... or removing it entirely (it takes up approx 1GB when compiled). If you don't want to keep sources then removing /usr/portage/distfiles/* will free that up, or see, 'eclean-dist' from app-portage/gentoolkit for managing distfiles. Also check you don't have any builds under /var/tmp/portage (failed, and or killed, builds leave their sources ... which in some cases can be space consuming). Note that package selection, and useflags, play some role in this ... but I'll leave such questions for another time.

best ... khay
Back to top
View user's profile Send private message
charles17
Advocate
Advocate


Joined: 02 Mar 2008
Posts: 2613

PostPosted: Fri Jan 15, 2016 4:14 pm    Post subject: Reply with quote

khayyam wrote:
@josephg ... did you remove the stage3 and portage-snapshot subsequent to unpacking?

khayyam
Downloading and unpacking a portage snapshot was in the past. Presently the official handbook recommends emerge-webrsync for getting the snapshot.
Back to top
View user's profile Send private message
khayyam
Watchman
Watchman


Joined: 07 Jun 2012
Posts: 6228
Location: Room 101

PostPosted: Fri Jan 15, 2016 7:28 pm    Post subject: Reply with quote

charles17 wrote:
khayyam wrote:
@josephg ... did you remove the stage3 and portage-snapshot subsequent to unpacking?

Downloading and unpacking a portage snapshot was in the past. Presently the official handbook recommends emerge-webrsync for getting the snapshot.

charles ... fair enough, but that still leaves the stage3.

As an aside, I can't see why this is change is recommended, you've just fetched the stage3 and so grabbing portage-snapshot isn't too much of an effort.

best ... khay
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43198
Location: 56N 3W

PostPosted: Fri Jan 15, 2016 7:35 pm    Post subject: Reply with quote

josephg,

/usr/portage/distfiles will grow without limit.
broken ebuilds accumulate in /var/tmp/portage
Kernel sources accumulate in /usr/src/
Portage build logs accumulate in /var/log/portage/ if you enable logging.

Code:
Filesystem                1K-blocks      Used Available Use% Mounted on
/dev/mapper/vg-distfiles       92760056  86643076   1697652  99% /usr/portage/distfiles

That's 86G of distfles since April 2009.

Gentoo needs more space that a binary distro.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
khayyam
Watchman
Watchman


Joined: 07 Jun 2012
Posts: 6228
Location: Room 101

PostPosted: Fri Jan 15, 2016 8:41 pm    Post subject: Reply with quote

NeddySeagoon wrote:
That's 86G of distfles since April 2009.

Neddy ... making you the go-to man for post-apocalypse survivors attempting to re-build the internet ;)

best ... khay
Back to top
View user's profile Send private message
Goverp
l33t
l33t


Joined: 07 Mar 2007
Posts: 668

PostPosted: Sat Jan 16, 2016 11:40 am    Post subject: Reply with quote

NeddySeagoon wrote:

...
/usr/portage/distfiles will grow without limit.
broken ebuilds accumulate in /var/tmp/portage
Kernel sources accumulate in /usr/src/
Portage build logs accumulate in /var/log/portage/ if you enable logging.
...

FWIW I configure make.conf to put distfiles (after all, it's just a cache), into /var/tmp, and have a cron job to manage that (it is temp space, so I clear files older than 1 month), and use elogviewer to manage build logs. emerge --depclean will clear out most of the old kernel sources - if anything, too aggressively, as it leaves only the most recent kernel emerged, whether or not it's actually installed.

My netbook currently uses 11G in total. Not sure where all that is going!
_________________
Greybeard
Back to top
View user's profile Send private message
szatox
Veteran
Veteran


Joined: 27 Aug 2013
Posts: 1746

PostPosted: Sat Jan 16, 2016 9:21 pm    Post subject: Reply with quote

Quote:
emerge --depclean will clear out most of the old kernel sources

Are you sure? I recently removed sources for 5 or 6 kernels. Each of those directories with already unmerged soures takes over 0.5GB
Code:
# du -csh /usr/src/*
4.0K   /usr/src/boot
0   /usr/src/linux
521M   /usr/src/linux-4.0.9-gentoo
1.2G   /usr/src/linux-4.1.12-gentoo


Also, the old kernel modules accumulate too. You can say it doesn't take a lot of space, but it does take space regardless:
Code:
# du -csh /lib/modules/*
55M   /lib/modules/3.10.17-gentoo
59M   /lib/modules/3.10.25-gentoo
58M   /lib/modules/3.10.7-gentoo
63M   /lib/modules/3.12.13-gentoo
62M   /lib/modules/3.12.20-gentoo
62M   /lib/modules/3.12.21-gentoo-r1
62M   /lib/modules/3.14.14-gentoo
63M   /lib/modules/3.16.5-gentoo
61M   /lib/modules/3.17.7-gentoo
65M   /lib/modules/3.17.8-gentoo-r1
66M   /lib/modules/3.18.12-gentoo
65M   /lib/modules/3.18.7-gentoo
50M   /lib/modules/3.6.11-gentoo
54M   /lib/modules/3.7.10-gentoo
54M   /lib/modules/3.7.10-gentoo-r1
48M   /lib/modules/3.7.9-gentoo
47M   /lib/modules/3.8.13
55M   /lib/modules/3.8.13-gentoo
69M   /lib/modules/4.0.5-gentoo
52M   /lib/modules/4.0.9-gentoo
51M   /lib/modules/4.1.12-gentoo
1.2G   total
Back to top
View user's profile Send private message
Tony0945
Advocate
Advocate


Joined: 25 Jul 2006
Posts: 3078
Location: Illinois, USA

PostPosted: Sun Jan 17, 2016 12:40 am    Post subject: Reply with quote

szatox wrote:
Quote:
emerge --depclean will clear out most of the old kernel sources

Are you sure? I recently removed sources for 5 or 6 kernels. Each of those directories with already unmerged soures takes over 0.5GB


emerge --depclean will remove the kernel sources from portage but does not erase the modules or the source code (nor any compiled object code).

There is an einfo message warning about this when you do the unmerge. You have to delete them by yourself.
Back to top
View user's profile Send private message
josephg
l33t
l33t


Joined: 10 Jan 2016
Posts: 783
Location: usually offline

PostPosted: Sun Jan 17, 2016 3:09 pm    Post subject: Reply with quote

khayyam wrote:
@josephg ... did you remove the stage3 and portage-snapshot subsequent to unpacking? You might also save some space by running 'make clean' on /usr/src/linux-{version} ... or removing it entirely (it takes up approx 1GB when compiled). If you don't want to keep sources then removing /usr/portage/distfiles/* will free that up, or see, 'eclean-dist' from app-portage/gentoolkit for managing distfiles. Also check you don't have any builds under /var/tmp/portage (failed, and or killed, builds leave their sources ... which in some cases can be space consuming). Note that package selection, and useflags, play some role in this ... but I'll leave such questions for another time.


yes i removed the initial downloads, as i assumed i wouldn't need them again. i might need to recompile my kernel everything doesn't look right. but i'll create another thread for that.
i see no need to keep all sources. i can always download them again when needed. compiliing takes a lot longer than downloading.
i'll also look at all those directories mentioned.
and i think i need to look at gentoolkit. i had put than on pending, thinking it wasn't core.
Back to top
View user's profile Send private message
josephg
l33t
l33t


Joined: 10 Jan 2016
Posts: 783
Location: usually offline

PostPosted: Sun Jan 17, 2016 3:17 pm    Post subject: Reply with quote

wow looking at some of your diskspace usage, i think i have vastly underestimated the needs of gentoo.. and i should probably dedicate some more diskspace than i had initially planned. but this first exercise with gentoo has helped me understand a lot more, thanks to you all. i have my system up & running now. but it doesn't feel clean. should i scratch this install, or can i salvage this?

i'm thinking i want to install from scratch again, the next time not dirtying my configs & installs like i did this time. what directories can i share with another install. some of you have mentioned the huge portage directories. would i need a clean slate, rather than dirtying a new install with no so clean configs from this time?
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43198
Location: 56N 3W

PostPosted: Sun Jan 17, 2016 3:56 pm    Post subject: Reply with quote

josephg,

There is no need to start over unless you make some gastly mistake or want to upgrade your hardware to some other arch.
Its quite common to migrate gentoo across hardware.

You can clean up. There are tools for some things ...
eclean will purge your distfiles and packages
emerge --depclean will remove orphaned packages from your live system.
Do look at what it wants to do first. A long time ago it removed my glibc.

Kernel sources cleaning is semi automated. emerge --depclean will leave all the binary files and your .config behind.
/boot needs to be cleaned by hand
/lib/modules/ needs to be cleaned by hand
/var/tmp/portage needs to be cleaned by hand.
Most logs can be rotated. How, depends on your logger.

You can add and migrate parts of your install across partitions too. There in no need to stick with the Handbook partition layout.
A separate /home partition is a good idea.
/var can be its own partition. /usr too but that may need an intrd to make work.

With regards sharing, /boot can be shared among distros. swap can be shared as long as you don't use hibernate. /tmp can be shared if its not in RAM anyway. Unless you are doing something unusual, /tmp is flushed at boot.

None of this needs a new install.
On the downside none of the above removes config files.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
josephg
l33t
l33t


Joined: 10 Jan 2016
Posts: 783
Location: usually offline

PostPosted: Sun Jan 17, 2016 4:07 pm    Post subject: Reply with quote

thank you neddy, i was having second thoughts. i'm a bit ocd about having a clean system. the only thing keeping me from trashing this install and starting another was the thought of the long times spent looking at compile output flying past. you've told me now how to --quiet that.

on the plus side, this is a working install. btrfs partition with two subvols. no boot errors. no systemd, no udev, no gnome bits & bobs littered all over. eudev seems to be working. working ethernet. lxqt running far quicker than arch/debian. and i'm using qupzilla to access this forum now.

i think i will keep going with this install, and try to clean up. which probably means i'm going to create a few more threads to disrupt the sanity here ;)


Last edited by josephg on Sun Jan 17, 2016 10:25 pm; edited 1 time in total
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43198
Location: 56N 3W

PostPosted: Sun Jan 17, 2016 4:23 pm    Post subject: Reply with quote

josephg,

Just in case the penny hasn't dropped yet. Gentoo is portage and the ebuild tree. Everything else is $UPSTREAM
You use portage and the ebuild tree to build your very own distro your way.

Unlike binary distros, no two Gentoo installs are identical.

There is sanity here?
It must be in OTW.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
josephg
l33t
l33t


Joined: 10 Jan 2016
Posts: 783
Location: usually offline

PostPosted: Sun Jan 17, 2016 5:24 pm    Post subject: Reply with quote

i knew that!! but i think the penny is dropping very gently ;)
Back to top
View user's profile Send private message
Goverp
l33t
l33t


Joined: 07 Mar 2007
Posts: 668

PostPosted: Mon Jan 18, 2016 10:25 am    Post subject: Reply with quote

For what little it's worth, I threw kdirstat at my netbook. It's a small KDE-based machine, though with Java, Firefox and Calligra.

I have about 500,000 files and directories, 10% of which are directories.
/usr takes ~5Gb, of which /usr/lib is 1Gb, as is /usr/src, /usr/share 800Mb, /usr/portage 500Mb
/home ~4.5 Gb, of which 40% comprises .cache and .local files.

The biggest application bits seem to be akonadi (KDE email search) at 300Mb, and clamav antivirus at 250Mb.
_________________
Greybeard
Back to top
View user's profile Send private message
josephg
l33t
l33t


Joined: 10 Jan 2016
Posts: 783
Location: usually offline

PostPosted: Mon Jan 18, 2016 10:38 pm    Post subject: Reply with quote

lol Goverp you call it small? :p kde java firefox calligra.. a 300mb application called akonadi.. another 250mb application called calmav

i'd be happy if my entire system fit in that space which those last two apps use. i've managed to keep within 1gb with wheezy and alpine, with everything i need.

me thinks my sense of minimalism is a bit different to you guys' ;)
Back to top
View user's profile Send private message
Goverp
l33t
l33t


Joined: 07 Mar 2007
Posts: 668

PostPosted: Tue Jan 19, 2016 9:39 am    Post subject: Reply with quote

Small in terms of number of packages in @world. Which was my point, a single application can take a whole load of space.

KDE comes with a massive amount of "environment", and I forgot to mention the full Eclipse for C++ development environment (256Mb)!

Mind you, having dug a bit deeper, I clawed back 2Gb of cruft - old cache and stuff from Chromium (I reverted to firefox) and a backup copy of akonadi data.
_________________
Greybeard
Back to top
View user's profile Send private message
cwr
Veteran
Veteran


Joined: 17 Dec 2005
Posts: 1969

PostPosted: Tue Jan 19, 2016 2:10 pm    Post subject: Reply with quote

I use 16 GB for /, though I've managed with 8 GB in the past, 16 GB for /home,
(music and VirtualBox images get separate partions under home), 8 GB for /var (too much),
16 GB for /usr/portage, and 256 MB for /boot. That's lasted me for a few years now.

Obviously that will vary depending on what packages you have installed. I have a full
Gnome 2.x environment and a fair number of applications.

Will
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43198
Location: 56N 3W

PostPosted: Tue Jan 19, 2016 6:28 pm    Post subject: Reply with quote

josephg,

Binary distros do not install source code, the tool chain and the build time dependencies are optional extras too.

On Gentoo, you must have the tool chain and build time dependencies that you need to use.
You also need to keep the source code, at least long enough to unpack it.
You need build space for packages to build. Libreoffice needs 6G of temporary space. There are bigger packages in overlays.
You also need install space, both in the target (installed and working) location and the build space as packages are installed into the build location then copied to the root filesystem.
The install space in the build location in included is the build space above.

It boils down to Gentoo needing more space that a binary install because you need to have more things installed.
Gentoo also needs temporary space that binary distros don't.

The temporary space can be in RAM too. With 4G RAM almost everything will build in the default 2G tmpfs.
If you want to be really frugal with your install space, the running (binary) part of Gentoo will fix in 6G. For my netbook, I build on a USB rotating rust drive and install on the internal 8G SSD.
There are no distfiles, no portage tree and no kernel sources on the SSD (the toolchain and build time dependencies are still there) but the netbook is running Gentoo when I am away from home.
I could remove more but it fits in 6G, so I don't need to.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
josephg
l33t
l33t


Joined: 10 Jan 2016
Posts: 783
Location: usually offline

PostPosted: Wed Jan 20, 2016 1:36 am    Post subject: Reply with quote

NeddySeagoon wrote:
The temporary space can be in RAM too. With 4G RAM almost everything will build in the default 2G tmpfs.

thanks for this tip :) i found how to do this on the wiki. i use tmpfs for practically everything i can, including ~/.cache

NeddySeagoon wrote:
If you want to be really frugal with your install space, the running (binary) part of Gentoo will fix in 6G. For my netbook, I build on a USB rotating rust drive and install on the internal 8G SSD.
There are no distfiles, no portage tree and no kernel sources on the SSD (the toolchain and build time dependencies are still there) but the netbook is running Gentoo when I am away from home.
I could remove more but it fits in 6G, so I don't need to.


how do you do this? or is there some article perhaps you could refer me to..

this is really what i'm aiming for. my frugal netbook doesn't need all that lot on travels, and i need some diskspace for work. when needed, i could perhaps sync (if i may use the term loosely) or install apps from my home computer which will have all the trees, sources, toolchains, distfiles, etc. without all these, i'm hoping the resulting gentoo (let's say netbook version?) would be much smaller compared to other binary distros.

ps: what is a "usb rotating rust drive"? google turned up nothing, except this thread!!
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43198
Location: 56N 3W

PostPosted: Wed Jan 20, 2016 6:52 pm    Post subject: Reply with quote

josephg,

The magnetic coating on traditional hard drives and floppy drives used to be an Iron Oxide compound.
One form of iron oxide is rust. Traditional hard drives hold the data on spinning platters, coated in magnetic material.
Hence "Rotating Rust". Its not actually true. Rust is the wrong iron oxide, but it serves to separate traditional HDD from Solid State HDD, which have no moving parts, except electrons.

My netbook install is maintained on a USB HDD. Its a traditional install, following the handbook with a few exceptions.
1. /usr/portage has its own partition.
2. Everything needed for USB Support is configured in the kernel as <*>, since I don't use an initrd and with root on USB, the kernel needs to be able to read USB storage all on its own.
3. The kernel command line contains root_delay=7. Normally, root is mounted before USB is started, so root on USB fails. root_delay=7 tells the kernel to wail 7 seconds before attempting to mount root. During this time the USB system becomes ready.

Everything just works off the USB HDD, which is the way I use it at my desk.

For travelling, I boot the netboot with a USB stick and make new /boot and root filesystems on the netbook internal drive.
Its about 32Mb for boot, swap=RAM, in case I want to play with hibernate and the rest for root. That gives me just over 6G for root from an 8G SSD.
I mount the USB install at /mnt/gentoo - read only ... In case I do the copy the wrong way but leave /usr/portage unmounted.
I mount the internal partitions at /mnt/new
Then cp -a one to the other. There are several ways to do the copy.
Now empty /usr/src because its not needed. Its a good idea to prune /usr/src to one kernel before you start.

Once the copy is complete, chroot into /mnt/new.
Fix /boot and /etc/fstab as root will have moved from sdb to sda. It may have changed partition number too.
localmount will complain about the /usr/portage filesysem not being found but that's cosmetic. Remove it from fstab if its annoying.
Reinstall the boot loader to the MBR. Mixing and matching bits of bootloader across different versions is usually a bad idea.

I leave the ssh host key, static IP, hostname and so on unchanged. This is harmless to me as both installs are never booted at the same time and both refer to the same machine anyway.
You need to consider your use case.

As long as you do the copy as root and preserve permissions, cp -a, tar and rsync all work.

Reboot to the internal HDD to test.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Goto page 1, 2, 3  Next
Page 1 of 3

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum