Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Why does /lib/modules take up so much space?
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
mi_unixbird
Tux's lil' helper
Tux's lil' helper


Joined: 24 Jul 2015
Posts: 102

PostPosted: Sun Jul 26, 2015 1:05 pm    Post subject: Why does /lib/modules take up so much space? Reply with quote

Code:

 —— — du -sBM /lib/modules/
2285M   /lib/modules/ # gentoo has 2.3 GB worth of modules
 —— — sudo mount /dev/sdb5 /mnt/mint # linux mint installation on the same drive
 —— — du -sBM /mnt/mint/lib/modules
226M    /mnt/mint/lib/modules # only 200 MB of kernel modules without a custom kernel ?
 —— — df -h
Filesystem      Size  Used Avail Use% Mounted on
udev             10M  4.0K   10M   1% /dev
/dev/sdb6        40G   11G   27G  29% / # gentoo install 11 GiB
tmpfs           788M  1.1M  787M   1% /run
shm             3.9G   98M  3.8G   3% /dev/shm
cgroup_root      10M     0   10M   0% /sys/fs/cgroup
/dev/sdb2       190M   97M   84M  54% /boot
/dev/sda1       688G   99G  554G  16% /home
/dev/sdb5        40G  9.2G   29G  25% /mnt/mint # mint install only 9.2 GiB


so what's th edeal here? I have a custom kernel and unchecked everything I remotely recnogized and knew I didn't have and some things I did have like ethernet cables. Did I do something wrongly?

Edit, also:

Code:
 —— — du -BM --max-depth=1 /lib/modules/4.0.5-gentoo/kernel/
1M      /lib/modules/4.0.5-gentoo/kernel/kernel
322M    /lib/modules/4.0.5-gentoo/kernel/net
1693M   /lib/modules/4.0.5-gentoo/kernel/drivers
121M    /lib/modules/4.0.5-gentoo/kernel/sound
1M      /lib/modules/4.0.5-gentoo/kernel/mm
7M      /lib/modules/4.0.5-gentoo/kernel/lib
16M     /lib/modules/4.0.5-gentoo/kernel/crypto
19M     /lib/modules/4.0.5-gentoo/kernel/arch
89M     /lib/modules/4.0.5-gentoo/kernel/fs
2265M   /lib/modules/4.0.5-gentoo/kernel/


Seems like kernel/drivers is the major culprit here, anything I did wrongly with the drivers?


Last edited by mi_unixbird on Sun Jul 26, 2015 1:29 pm; edited 2 times in total
Back to top
View user's profile Send private message
Buffoon
Veteran
Veteran


Joined: 17 Jun 2015
Posts: 1074
Location: EU or US

PostPosted: Sun Jul 26, 2015 1:17 pm    Post subject: Reply with quote

My modules are 26 MB, but I clean it up every now and then. And I build majority of functionality into kernel.
Back to top
View user's profile Send private message
mi_unixbird
Tux's lil' helper
Tux's lil' helper


Joined: 24 Jul 2015
Posts: 102

PostPosted: Sun Jul 26, 2015 1:26 pm    Post subject: Reply with quote

How do you clean it up? Is there some functionality for that?
Back to top
View user's profile Send private message
kite14
Apprentice
Apprentice


Joined: 07 Nov 2006
Posts: 199
Location: Pordenone/Italy

PostPosted: Sun Jul 26, 2015 1:47 pm    Post subject: Reply with quote

mi_unixbird wrote:
How do you clean it up? Is there some functionality for that?

I'm not aware of any tools for this job; I usually clean it manually by removing old
kernels and related files in /boot folder and then the corresponding folder in /lib/modules
.
Let's say you want to get rid of 3.x.x kernels (and modules):
1. mount /boot (if it's in a partition of its own)
2. rm /boot/*-3.* (delete vmlinuz, config and system.map files of 3.x.x kernels)
3. rm -r /lib/modules/3.*

For example, my 2 years-old laptop only has a couple of kernel images to boot from:
Code:
ls -l /boot
totale 14555
-rw-r--r-- 1 root root 2468638 14 mag 12.43 System.map-3.18.12-gentoo
-rw-r--r-- 1 root root 2524522 21 giu 23.03 System.map-4.0.5-gentoo
-rw-r--r-- 1 root root   92803 14 mag 12.43 config-3.18.12-gentoo
-rw-r--r-- 1 root root   94313 21 giu 23.03 config-4.0.5-gentoo
drwxr-xr-x 6 root root    1024 30 giu 00.03 grub
drwx------ 2 root root   12288 30 dic  2013 lost+found
drwxr-xr-x 2 root root    1024 31 ott  2014 memtest86plus
-rw-r--r-- 1 root root 4775952 14 mag 12.43 vmlinuz-3.18.12-gentoo
-rw-r--r-- 1 root root 4864784 21 giu 23.03 vmlinuz-4.0.5-gentoo

Code:
ls -l /lib/modules
totale 1
drwxr-xr-x 3 root root 496 14 mag 12.42 3.18.12-gentoo
drwxr-xr-x 3 root root 496 21 giu 23.03 4.0.5-gentoo

Code:
du -sBM /lib/modules
20M     /lib/modules


Last edited by kite14 on Sun Jul 26, 2015 1:55 pm; edited 1 time in total
Back to top
View user's profile Send private message
mi_unixbird
Tux's lil' helper
Tux's lil' helper


Joined: 24 Jul 2015
Posts: 102

PostPosted: Sun Jul 26, 2015 1:53 pm    Post subject: Reply with quote

Ah, well, no, these are the modules for the current version only, in particular kernel/drivers seem to take up the vast majority of the space, anything I did wrong during compilation?

Code:
 —— — du -BM --max-depth=1 /lib/modules/4.0.5-gentoo/kernel/drivers/ | sort -rg | head -10
1693M   /lib/modules/4.0.5-gentoo/kernel/drivers/
277M    /lib/modules/4.0.5-gentoo/kernel/drivers/net
259M    /lib/modules/4.0.5-gentoo/kernel/drivers/media
245M    /lib/modules/4.0.5-gentoo/kernel/drivers/staging
193M    /lib/modules/4.0.5-gentoo/kernel/drivers/gpu
115M    /lib/modules/4.0.5-gentoo/kernel/drivers/scsi
71M     /lib/modules/4.0.5-gentoo/kernel/drivers/usb
64M     /lib/modules/4.0.5-gentoo/kernel/drivers/infiniband
42M     /lib/modules/4.0.5-gentoo/kernel/drivers/video
37M     /lib/modules/4.0.5-gentoo/kernel/drivers/hwmon


I guess I'm supposed to find answers in net, media, gpu and stagin?
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6281

PostPosted: Sun Jul 26, 2015 2:03 pm    Post subject: Reply with quote

These numbers are really unbelievable. For instance, I have only 8MB (though xz-compressed, but neveretheless...).
Do you have only *.ko files in there?
Even if you do strange this like adding debugging information, I think that it is almost impossible to come to >99MB altogether for one kernel.

The nvidia graphics driver (if you use it) takes a lot of space, but everythnig else should be small...
Back to top
View user's profile Send private message
kite14
Apprentice
Apprentice


Joined: 07 Nov 2006
Posts: 199
Location: Pordenone/Italy

PostPosted: Sun Jul 26, 2015 2:26 pm    Post subject: Reply with quote

Oh, sorry... I didn't pay attention to the second code block in your first post.
Where did you start to strip down your .config from? I used to start from one of
pappy's kernel seed and never came across such a problem.
Back to top
View user's profile Send private message
mi_unixbird
Tux's lil' helper
Tux's lil' helper


Joined: 24 Jul 2015
Posts: 102

PostPosted: Sun Jul 26, 2015 2:27 pm    Post subject: Reply with quote

mv wrote:
These numbers are really unbelievable. For instance, I have only 8MB (though xz-compressed, but neveretheless...).
Do you have only *.ko files in there?
Even if you do strange this like adding debugging information, I think that it is almost impossible to come to >99MB altogether for one kernel.

The nvidia graphics driver (if you use it) takes a lot of space, but everythnig else should be small...


Yeah, that was my idea too, that something was wrong.

Code:
 —— — find /lib/modules/4.0.5-gentoo/kernel/drivers/ -type f | egrep -v '\.ko$'
 —— 1


The one is an error code, as in "nothing found" every standard file in there ends on *.ko it seems.

I do use the proprietary driver so I accept the size of drivers/gpu but everything else seems waaay too large,
Back to top
View user's profile Send private message
mi_unixbird
Tux's lil' helper
Tux's lil' helper


Joined: 24 Jul 2015
Posts: 102

PostPosted: Sun Jul 26, 2015 2:30 pm    Post subject: Reply with quote

kite14 wrote:
Oh, sorry... I didn't pay attention to the second code block in your first post.
Where did you start to strip down your .config from? I used to start from one of
pappy's kernel seed and never came across such a problem.


I started simply by using
Code:
emerge gentoo-sources
and then worked my way to the source directory and hit
Code:
make menuconfig


Is it normal for that to have such a huge set of kernel modules? The kernel image itself is 7MB, same as the Ubuntu one.
Back to top
View user's profile Send private message
mi_unixbird
Tux's lil' helper
Tux's lil' helper


Joined: 24 Jul 2015
Posts: 102

PostPosted: Sun Jul 26, 2015 2:35 pm    Post subject: Reply with quote

Looking into .config manually by the way, no line ends on =n, is this normal or are the not included lines just omitted? Everything is either =m or =y.
Back to top
View user's profile Send private message
kite14
Apprentice
Apprentice


Joined: 07 Nov 2006
Posts: 199
Location: Pordenone/Italy

PostPosted: Sun Jul 26, 2015 2:39 pm    Post subject: Reply with quote

mi_unixbird wrote:
Looking into .config manually by the way, no line ends on =n, is this normal or are the not included lines just omitted? Everything is either =m or =y.

They are not omittted, but are commented out like this:
Code:
# CONFIG_<MODULE_NAME> is not set
Back to top
View user's profile Send private message
kite14
Apprentice
Apprentice


Joined: 07 Nov 2006
Posts: 199
Location: Pordenone/Italy

PostPosted: Sun Jul 26, 2015 3:09 pm    Post subject: Reply with quote

How many modules have been enabled in your .config?
My .config:
Code:
cat /boot/config-4.0.5-gentoo | grep "=m" | wc -l
99

Code:
find /lib/modules/4.0.5-gentoo/kernel/drivers/ -type f | egrep  '\.ko$' | wc -l
58
Back to top
View user's profile Send private message
mi_unixbird
Tux's lil' helper
Tux's lil' helper


Joined: 24 Jul 2015
Posts: 102

PostPosted: Sun Jul 26, 2015 3:17 pm    Post subject: Reply with quote

kite14 wrote:
How many modules have been enabled in your .config?
My .config:
Code:
cat /boot/config-4.0.5-gentoo | grep "=m" | wc -l
99

Code:
find /lib/modules/4.0.5-gentoo/kernel/drivers/ -type f | egrep  '\.ko$' | wc -l
58


Code:
 —— — cat /boot/config-4.0.5-gentoo | grep "=m" | wc -l
3620


Yees, well. I guess that's part of the problem. I pretty much disabled a lot of the stuff though.

But anyway, I've followed the kernel seed instructions and managed to disable a lot more using it, we'll see how much this compilation gives.

For comparison though:

Code:
 —— — cat /boot/config-4.0.6-040006-generic | grep "=m" | wc -l
4132


Which is the Ubuntu kernel with waaaaaaaaaaaaaaaay less space in /lib/modules


Well, after a new menuconfig I managed to bring it down about 500MiB and reduced the number of modules to around 2550, I still wonder where the hell most of those modules are at. THis started by unchecking the default menuconfig that came with "emerge gentoo-sources"
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 13831

PostPosted: Sun Jul 26, 2015 4:17 pm    Post subject: Reply with quote

For future reference, that pipeline is inefficient. You can feed the file directly to grep, and if all you want is a line count, grep can do that too.

Very bad: cat file | grep expr | wc -l
Less bad: grep expr file | wc -l
Good: grep -c expr file
Back to top
View user's profile Send private message
ct85711
Veteran
Veteran


Joined: 27 Sep 2005
Posts: 1696

PostPosted: Sun Jul 26, 2015 5:41 pm    Post subject: Reply with quote

I do have to say, that is a rather big modules directory, especialy just for one kernel. On my system, I get this

Code:
ct85711@Oate /lib/modules $ du -BM --max-depth=1
23M   ./3.15.0-gentoo-r1
31M   ./3.18.5-gentoo
24M   ./3.15.1-gentoo
41M   ./4.1.1-gentoo-r1
14M   ./3.6.6-gentoo
25M   ./3.15.7-gentoo
30M   ./3.5.1-gentoo
19M   ./3.10.5-gentoo-r1
13M   ./3.8.8-gentoo
26M   ./3.17.1-gentoo-r1
31M   ./3.18.1-gentoo
29M   ./3.5.0-gentoo
15M   ./3.9.7-gentoo
14M   ./3.7.2-gentoo
20M   ./3.12.6-gentoo
20M   ./3.14.5-gentoo-r1
13M   ./3.9.6-gentoo
25M   ./3.13.1-gentoo
407M   .
ct85711@Oate /lib/modules $ uname -a
Linux Oate 4.1.1-gentoo-r1 #2 SMP Fri Jul 10 23:05:54 CDT 2015 x86_64 AMD A10-7850K Radeon R7, 12 Compute Cores 4C+8G AuthenticAMD GNU/Linux


I've always used gentoo-sources to compiled all my kernels, even with the ATI/NVIDIA drivers my modules for any one kernel never grew more than 50M (yes, some of my kernels has both the ati and nvidia drivers in the modules, as I switched form using the OBG ATI to a discrete nvidia card). That's also with the virtualbox drivers too. My current modules directory grew mostly because I added quite a few additional modules, which I could easily go back through and shrink back down if I wanted to.

Note: I haven't cleaned my modules directory ever, but I haven't seen a need to either, since my current 400MB is nothing compared to my home directory taking multi GB.
Back to top
View user's profile Send private message
mi_unixbird
Tux's lil' helper
Tux's lil' helper


Joined: 24 Jul 2015
Posts: 102

PostPosted: Sun Jul 26, 2015 6:01 pm    Post subject: Reply with quote

Yeah, so, I found out that even if I completely delete (back up) the /usr/src/linux-4.0.5-gentoo folder, remerge the sources and do "make menuconfig" it plants a .config identical to the old one that is on some remote part on another filesystem in my home directory. So I guess the reason it's so bloated is because it found something weird from another kernel. I installed Gentoo not from a live environment but from a working debian one in xorg which has given me more problems than it was worth thusfar I have to say, the installation guide while saying it is possible often seems to assume you use a minimal live environment, some parts of the actual working environment do pollute it.

So I did "make defconfig", guess what, only 17 modules now and 50 MiB now.

Code:

 —— — du -sBM /lib/modules/
12M     /lib/modules/


I have no idea what was in that other config. Like, I unchecked like everything I could find at the end and the modules directory was still 15GiB.
Back to top
View user's profile Send private message
jonathan183
Guru
Guru


Joined: 13 Dec 2011
Posts: 309

PostPosted: Sun Jul 26, 2015 6:35 pm    Post subject: Reply with quote

It has nothing to do with the running kernel or another distro installed on a separate partition.

The config is pulled in when you emerge the kernel sources, if you already have a .config in the folder it will not be overwritten by default when you re-emerge.

It sounds as though you are much closer to what you require ... personally I prefer to only enable the things I need, if it's on new hardware then sometimes I'll emerge genkernel and then build a manually configured kernel once I get the system to boot. It also gives me a fallback kernel to boot from incase I manage to screw up the current kernel config
Back to top
View user's profile Send private message
mi_unixbird
Tux's lil' helper
Tux's lil' helper


Joined: 24 Jul 2015
Posts: 102

PostPosted: Sun Jul 26, 2015 7:05 pm    Post subject: Reply with quote

jonathan183 wrote:
It has nothing to do with the running kernel or another distro installed on a separate partition.

The config is pulled in when you emerge the kernel sources, if you already have a .config in the folder it will not be overwritten by default when you re-emerge.
Like I said, I moved the entire /usr/src/linux-4.0.5-gentoo folder to another physical drive. Then I emerged the sources, then I checked if a .config file existed, which it did not, then I did "make menuconfig" and that program made an exact copy of my old config. Maybe it fetches it from /boot or something?
Back to top
View user's profile Send private message
kite14
Apprentice
Apprentice


Joined: 07 Nov 2006
Posts: 199
Location: Pordenone/Italy

PostPosted: Sun Jul 26, 2015 8:20 pm    Post subject: Reply with quote

Hu wrote:
For future reference, that pipeline is inefficient. You can feed the file directly to grep, and if all you want is a line count, grep can do that too.

Very bad: cat file | grep expr | wc -l
Less bad: grep expr file | wc -l
Good: grep -c expr file

Thanks for your suggestion, noted down for future reference.
Back to top
View user's profile Send private message
mi_unixbird
Tux's lil' helper
Tux's lil' helper


Joined: 24 Jul 2015
Posts: 102

PostPosted: Sun Jul 26, 2015 9:32 pm    Post subject: Reply with quote

kite14 wrote:
Hu wrote:
For future reference, that pipeline is inefficient. You can feed the file directly to grep, and if all you want is a line count, grep can do that too.

Very bad: cat file | grep expr | wc -l
Less bad: grep expr file | wc -l
Good: grep -c expr file

Thanks for your suggestion, noted down for future reference.
Hey, what's important is that your version embodied the Unix philosophy. grep is a total monolithic block. It reads from a file, it selects the lines that match a pattern and it counts as well. Won't be long before we're basically using Windows and it's tied into the kernel not allowing competing implementations.
Back to top
View user's profile Send private message
kite14
Apprentice
Apprentice


Joined: 07 Nov 2006
Posts: 199
Location: Pordenone/Italy

PostPosted: Sun Jul 26, 2015 10:39 pm    Post subject: Reply with quote

mi_unixbird wrote:
Won't be long before we're basically using Windows...

Or, most likely, a systemd implementation of grep.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum