View previous topic :: View next topic |
Author |
Message |
mi_unixbird Tux's lil' helper


Joined: 24 Jul 2015 Posts: 102
|
Posted: Sun Jul 26, 2015 1:05 pm Post subject: Why does /lib/modules take up so much space? |
|
|
Code: |
—— — du -sBM /lib/modules/
2285M /lib/modules/ # gentoo has 2.3 GB worth of modules
—— — sudo mount /dev/sdb5 /mnt/mint # linux mint installation on the same drive
—— — du -sBM /mnt/mint/lib/modules
226M /mnt/mint/lib/modules # only 200 MB of kernel modules without a custom kernel ?
—— — df -h
Filesystem Size Used Avail Use% Mounted on
udev 10M 4.0K 10M 1% /dev
/dev/sdb6 40G 11G 27G 29% / # gentoo install 11 GiB
tmpfs 788M 1.1M 787M 1% /run
shm 3.9G 98M 3.8G 3% /dev/shm
cgroup_root 10M 0 10M 0% /sys/fs/cgroup
/dev/sdb2 190M 97M 84M 54% /boot
/dev/sda1 688G 99G 554G 16% /home
/dev/sdb5 40G 9.2G 29G 25% /mnt/mint # mint install only 9.2 GiB
|
so what's th edeal here? I have a custom kernel and unchecked everything I remotely recnogized and knew I didn't have and some things I did have like ethernet cables. Did I do something wrongly?
Edit, also:
Code: | —— — du -BM --max-depth=1 /lib/modules/4.0.5-gentoo/kernel/
1M /lib/modules/4.0.5-gentoo/kernel/kernel
322M /lib/modules/4.0.5-gentoo/kernel/net
1693M /lib/modules/4.0.5-gentoo/kernel/drivers
121M /lib/modules/4.0.5-gentoo/kernel/sound
1M /lib/modules/4.0.5-gentoo/kernel/mm
7M /lib/modules/4.0.5-gentoo/kernel/lib
16M /lib/modules/4.0.5-gentoo/kernel/crypto
19M /lib/modules/4.0.5-gentoo/kernel/arch
89M /lib/modules/4.0.5-gentoo/kernel/fs
2265M /lib/modules/4.0.5-gentoo/kernel/ |
Seems like kernel/drivers is the major culprit here, anything I did wrongly with the drivers?
Last edited by mi_unixbird on Sun Jul 26, 2015 1:29 pm; edited 2 times in total |
|
Back to top |
|
 |
Buffoon Veteran


Joined: 17 Jun 2015 Posts: 1074 Location: EU or US
|
Posted: Sun Jul 26, 2015 1:17 pm Post subject: |
|
|
My modules are 26 MB, but I clean it up every now and then. And I build majority of functionality into kernel. |
|
Back to top |
|
 |
mi_unixbird Tux's lil' helper


Joined: 24 Jul 2015 Posts: 102
|
Posted: Sun Jul 26, 2015 1:26 pm Post subject: |
|
|
How do you clean it up? Is there some functionality for that? |
|
Back to top |
|
 |
kite14 Apprentice


Joined: 07 Nov 2006 Posts: 199 Location: Pordenone/Italy
|
Posted: Sun Jul 26, 2015 1:47 pm Post subject: |
|
|
mi_unixbird wrote: | How do you clean it up? Is there some functionality for that? |
I'm not aware of any tools for this job; I usually clean it manually by removing old
kernels and related files in /boot folder and then the corresponding folder in /lib/modules
.
Let's say you want to get rid of 3.x.x kernels (and modules):
1. mount /boot (if it's in a partition of its own)
2. rm /boot/*-3.* (delete vmlinuz, config and system.map files of 3.x.x kernels)
3. rm -r /lib/modules/3.*
For example, my 2 years-old laptop only has a couple of kernel images to boot from:
Code: | ls -l /boot
totale 14555
-rw-r--r-- 1 root root 2468638 14 mag 12.43 System.map-3.18.12-gentoo
-rw-r--r-- 1 root root 2524522 21 giu 23.03 System.map-4.0.5-gentoo
-rw-r--r-- 1 root root 92803 14 mag 12.43 config-3.18.12-gentoo
-rw-r--r-- 1 root root 94313 21 giu 23.03 config-4.0.5-gentoo
drwxr-xr-x 6 root root 1024 30 giu 00.03 grub
drwx------ 2 root root 12288 30 dic 2013 lost+found
drwxr-xr-x 2 root root 1024 31 ott 2014 memtest86plus
-rw-r--r-- 1 root root 4775952 14 mag 12.43 vmlinuz-3.18.12-gentoo
-rw-r--r-- 1 root root 4864784 21 giu 23.03 vmlinuz-4.0.5-gentoo |
Code: | ls -l /lib/modules
totale 1
drwxr-xr-x 3 root root 496 14 mag 12.42 3.18.12-gentoo
drwxr-xr-x 3 root root 496 21 giu 23.03 4.0.5-gentoo |
Code: | du -sBM /lib/modules
20M /lib/modules |
Last edited by kite14 on Sun Jul 26, 2015 1:55 pm; edited 1 time in total |
|
Back to top |
|
 |
mi_unixbird Tux's lil' helper


Joined: 24 Jul 2015 Posts: 102
|
Posted: Sun Jul 26, 2015 1:53 pm Post subject: |
|
|
Ah, well, no, these are the modules for the current version only, in particular kernel/drivers seem to take up the vast majority of the space, anything I did wrong during compilation?
Code: | —— — du -BM --max-depth=1 /lib/modules/4.0.5-gentoo/kernel/drivers/ | sort -rg | head -10
1693M /lib/modules/4.0.5-gentoo/kernel/drivers/
277M /lib/modules/4.0.5-gentoo/kernel/drivers/net
259M /lib/modules/4.0.5-gentoo/kernel/drivers/media
245M /lib/modules/4.0.5-gentoo/kernel/drivers/staging
193M /lib/modules/4.0.5-gentoo/kernel/drivers/gpu
115M /lib/modules/4.0.5-gentoo/kernel/drivers/scsi
71M /lib/modules/4.0.5-gentoo/kernel/drivers/usb
64M /lib/modules/4.0.5-gentoo/kernel/drivers/infiniband
42M /lib/modules/4.0.5-gentoo/kernel/drivers/video
37M /lib/modules/4.0.5-gentoo/kernel/drivers/hwmon |
I guess I'm supposed to find answers in net, media, gpu and stagin? |
|
Back to top |
|
 |
mv Watchman


Joined: 20 Apr 2005 Posts: 6296
|
Posted: Sun Jul 26, 2015 2:03 pm Post subject: |
|
|
These numbers are really unbelievable. For instance, I have only 8MB (though xz-compressed, but neveretheless...).
Do you have only *.ko files in there?
Even if you do strange this like adding debugging information, I think that it is almost impossible to come to >99MB altogether for one kernel.
The nvidia graphics driver (if you use it) takes a lot of space, but everythnig else should be small... |
|
Back to top |
|
 |
kite14 Apprentice


Joined: 07 Nov 2006 Posts: 199 Location: Pordenone/Italy
|
Posted: Sun Jul 26, 2015 2:26 pm Post subject: |
|
|
Oh, sorry... I didn't pay attention to the second code block in your first post.
Where did you start to strip down your .config from? I used to start from one of
pappy's kernel seed and never came across such a problem. |
|
Back to top |
|
 |
mi_unixbird Tux's lil' helper


Joined: 24 Jul 2015 Posts: 102
|
Posted: Sun Jul 26, 2015 2:27 pm Post subject: |
|
|
mv wrote: | These numbers are really unbelievable. For instance, I have only 8MB (though xz-compressed, but neveretheless...).
Do you have only *.ko files in there?
Even if you do strange this like adding debugging information, I think that it is almost impossible to come to >99MB altogether for one kernel.
The nvidia graphics driver (if you use it) takes a lot of space, but everythnig else should be small... |
Yeah, that was my idea too, that something was wrong.
Code: | —— — find /lib/modules/4.0.5-gentoo/kernel/drivers/ -type f | egrep -v '\.ko$'
—— 1
|
The one is an error code, as in "nothing found" every standard file in there ends on *.ko it seems.
I do use the proprietary driver so I accept the size of drivers/gpu but everything else seems waaay too large, |
|
Back to top |
|
 |
mi_unixbird Tux's lil' helper


Joined: 24 Jul 2015 Posts: 102
|
Posted: Sun Jul 26, 2015 2:30 pm Post subject: |
|
|
kite14 wrote: | Oh, sorry... I didn't pay attention to the second code block in your first post.
Where did you start to strip down your .config from? I used to start from one of
pappy's kernel seed and never came across such a problem. |
I started simply by using Code: | emerge gentoo-sources | and then worked my way to the source directory and hit
Is it normal for that to have such a huge set of kernel modules? The kernel image itself is 7MB, same as the Ubuntu one. |
|
Back to top |
|
 |
mi_unixbird Tux's lil' helper


Joined: 24 Jul 2015 Posts: 102
|
Posted: Sun Jul 26, 2015 2:35 pm Post subject: |
|
|
Looking into .config manually by the way, no line ends on =n, is this normal or are the not included lines just omitted? Everything is either =m or =y. |
|
Back to top |
|
 |
kite14 Apprentice


Joined: 07 Nov 2006 Posts: 199 Location: Pordenone/Italy
|
Posted: Sun Jul 26, 2015 2:39 pm Post subject: |
|
|
mi_unixbird wrote: | Looking into .config manually by the way, no line ends on =n, is this normal or are the not included lines just omitted? Everything is either =m or =y. |
They are not omittted, but are commented out like this:
Code: | # CONFIG_<MODULE_NAME> is not set |
|
|
Back to top |
|
 |
kite14 Apprentice


Joined: 07 Nov 2006 Posts: 199 Location: Pordenone/Italy
|
Posted: Sun Jul 26, 2015 3:09 pm Post subject: |
|
|
How many modules have been enabled in your .config?
My .config:
Code: | cat /boot/config-4.0.5-gentoo | grep "=m" | wc -l
99 |
Code: | find /lib/modules/4.0.5-gentoo/kernel/drivers/ -type f | egrep '\.ko$' | wc -l
58 |
|
|
Back to top |
|
 |
mi_unixbird Tux's lil' helper


Joined: 24 Jul 2015 Posts: 102
|
Posted: Sun Jul 26, 2015 3:17 pm Post subject: |
|
|
kite14 wrote: | How many modules have been enabled in your .config?
My .config:
Code: | cat /boot/config-4.0.5-gentoo | grep "=m" | wc -l
99 |
Code: | find /lib/modules/4.0.5-gentoo/kernel/drivers/ -type f | egrep '\.ko$' | wc -l
58 |
|
Code: | —— — cat /boot/config-4.0.5-gentoo | grep "=m" | wc -l
3620
|
Yees, well. I guess that's part of the problem. I pretty much disabled a lot of the stuff though.
But anyway, I've followed the kernel seed instructions and managed to disable a lot more using it, we'll see how much this compilation gives.
For comparison though:
Code: | —— — cat /boot/config-4.0.6-040006-generic | grep "=m" | wc -l
4132
|
Which is the Ubuntu kernel with waaaaaaaaaaaaaaaay less space in /lib/modules
Well, after a new menuconfig I managed to bring it down about 500MiB and reduced the number of modules to around 2550, I still wonder where the hell most of those modules are at. THis started by unchecking the default menuconfig that came with "emerge gentoo-sources" |
|
Back to top |
|
 |
Hu Moderator

Joined: 06 Mar 2007 Posts: 14372
|
Posted: Sun Jul 26, 2015 4:17 pm Post subject: |
|
|
For future reference, that pipeline is inefficient. You can feed the file directly to grep, and if all you want is a line count, grep can do that too.
Very bad: cat file | grep expr | wc -l
Less bad: grep expr file | wc -l
Good: grep -c expr file |
|
Back to top |
|
 |
ct85711 Veteran

Joined: 27 Sep 2005 Posts: 1725
|
Posted: Sun Jul 26, 2015 5:41 pm Post subject: |
|
|
I do have to say, that is a rather big modules directory, especialy just for one kernel. On my system, I get this
Code: | ct85711@Oate /lib/modules $ du -BM --max-depth=1
23M ./3.15.0-gentoo-r1
31M ./3.18.5-gentoo
24M ./3.15.1-gentoo
41M ./4.1.1-gentoo-r1
14M ./3.6.6-gentoo
25M ./3.15.7-gentoo
30M ./3.5.1-gentoo
19M ./3.10.5-gentoo-r1
13M ./3.8.8-gentoo
26M ./3.17.1-gentoo-r1
31M ./3.18.1-gentoo
29M ./3.5.0-gentoo
15M ./3.9.7-gentoo
14M ./3.7.2-gentoo
20M ./3.12.6-gentoo
20M ./3.14.5-gentoo-r1
13M ./3.9.6-gentoo
25M ./3.13.1-gentoo
407M .
ct85711@Oate /lib/modules $ uname -a
Linux Oate 4.1.1-gentoo-r1 #2 SMP Fri Jul 10 23:05:54 CDT 2015 x86_64 AMD A10-7850K Radeon R7, 12 Compute Cores 4C+8G AuthenticAMD GNU/Linux
|
I've always used gentoo-sources to compiled all my kernels, even with the ATI/NVIDIA drivers my modules for any one kernel never grew more than 50M (yes, some of my kernels has both the ati and nvidia drivers in the modules, as I switched form using the OBG ATI to a discrete nvidia card). That's also with the virtualbox drivers too. My current modules directory grew mostly because I added quite a few additional modules, which I could easily go back through and shrink back down if I wanted to.
Note: I haven't cleaned my modules directory ever, but I haven't seen a need to either, since my current 400MB is nothing compared to my home directory taking multi GB. |
|
Back to top |
|
 |
mi_unixbird Tux's lil' helper


Joined: 24 Jul 2015 Posts: 102
|
Posted: Sun Jul 26, 2015 6:01 pm Post subject: |
|
|
Yeah, so, I found out that even if I completely delete (back up) the /usr/src/linux-4.0.5-gentoo folder, remerge the sources and do "make menuconfig" it plants a .config identical to the old one that is on some remote part on another filesystem in my home directory. So I guess the reason it's so bloated is because it found something weird from another kernel. I installed Gentoo not from a live environment but from a working debian one in xorg which has given me more problems than it was worth thusfar I have to say, the installation guide while saying it is possible often seems to assume you use a minimal live environment, some parts of the actual working environment do pollute it.
So I did "make defconfig", guess what, only 17 modules now and 50 MiB now.
Code: |
—— — du -sBM /lib/modules/
12M /lib/modules/
|
I have no idea what was in that other config. Like, I unchecked like everything I could find at the end and the modules directory was still 15GiB. |
|
Back to top |
|
 |
jonathan183 Guru

Joined: 13 Dec 2011 Posts: 309
|
Posted: Sun Jul 26, 2015 6:35 pm Post subject: |
|
|
It has nothing to do with the running kernel or another distro installed on a separate partition.
The config is pulled in when you emerge the kernel sources, if you already have a .config in the folder it will not be overwritten by default when you re-emerge.
It sounds as though you are much closer to what you require ... personally I prefer to only enable the things I need, if it's on new hardware then sometimes I'll emerge genkernel and then build a manually configured kernel once I get the system to boot. It also gives me a fallback kernel to boot from incase I manage to screw up the current kernel config |
|
Back to top |
|
 |
mi_unixbird Tux's lil' helper


Joined: 24 Jul 2015 Posts: 102
|
Posted: Sun Jul 26, 2015 7:05 pm Post subject: |
|
|
jonathan183 wrote: | It has nothing to do with the running kernel or another distro installed on a separate partition.
The config is pulled in when you emerge the kernel sources, if you already have a .config in the folder it will not be overwritten by default when you re-emerge. | Like I said, I moved the entire /usr/src/linux-4.0.5-gentoo folder to another physical drive. Then I emerged the sources, then I checked if a .config file existed, which it did not, then I did "make menuconfig" and that program made an exact copy of my old config. Maybe it fetches it from /boot or something? |
|
Back to top |
|
 |
kite14 Apprentice


Joined: 07 Nov 2006 Posts: 199 Location: Pordenone/Italy
|
Posted: Sun Jul 26, 2015 8:20 pm Post subject: |
|
|
Hu wrote: | For future reference, that pipeline is inefficient. You can feed the file directly to grep, and if all you want is a line count, grep can do that too.
Very bad: cat file | grep expr | wc -l
Less bad: grep expr file | wc -l
Good: grep -c expr file |
Thanks for your suggestion, noted down for future reference. |
|
Back to top |
|
 |
mi_unixbird Tux's lil' helper


Joined: 24 Jul 2015 Posts: 102
|
Posted: Sun Jul 26, 2015 9:32 pm Post subject: |
|
|
kite14 wrote: | Hu wrote: | For future reference, that pipeline is inefficient. You can feed the file directly to grep, and if all you want is a line count, grep can do that too.
Very bad: cat file | grep expr | wc -l
Less bad: grep expr file | wc -l
Good: grep -c expr file |
Thanks for your suggestion, noted down for future reference. | Hey, what's important is that your version embodied the Unix philosophy. grep is a total monolithic block. It reads from a file, it selects the lines that match a pattern and it counts as well. Won't be long before we're basically using Windows and it's tied into the kernel not allowing competing implementations. |
|
Back to top |
|
 |
kite14 Apprentice


Joined: 07 Nov 2006 Posts: 199 Location: Pordenone/Italy
|
Posted: Sun Jul 26, 2015 10:39 pm Post subject: |
|
|
mi_unixbird wrote: | Won't be long before we're basically using Windows... |
Or, most likely, a systemd implementation of grep. |
|
Back to top |
|
 |
|