Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[SOLVED] Too slow to load modules when booting
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
toku
n00b
n00b


Joined: 12 Jan 2019
Posts: 8

PostPosted: Fri Dec 27, 2019 1:55 am    Post subject: [SOLVED] Too slow to load modules when booting Reply with quote

Hello, everyone!

Recently booting Gentoo has become slow. I noticed that when booting, Genkernel said "Root block device found, skipping loading of module group "foo" ...

Code:
>> Genkernel 4.0.1 (2019-12-23 13:02:51 UTC). Linux kernel 5.4.6-gentoo
>> Activating mdev ...
>> Loading modules ...
    :: Loading from virtio:
    :: Loading from hyperv:
    :: Loading from nvme: Root block device found, skipping loading of module group "nvme" ...
    :: Loading from sata: Root block device found, skipping loading of module group "sata" ...
    :: Loading from scsi: Root block device found, skipping loading of module group "scsi" ...
    :: Loading from pata: Root block device found, skipping loading of module group "pata" ...
    :: Loading from usb: Root block device found, skipping loading of module group "usb" ...
    :: Loading from firewire: Root block device found, skipping loading of module group "firewire" ...
    :: Loading from waitscan: Root block device found, skipping loading of module group "waitscan" ...
    :: Loading from lvm: Root block device found, skipping loading of module group "lvm" ...
    :: Loading from dmraid: Root block device found, skipping loading of module group "dmraid" ...
    :: Loading from mdadm: Root block device found, skipping loading of module group "mdadm" ...
    :: Loading from fs:
    :: Loading from crypto: Root block device found, skipping loading of module group "crypto" ...
>> Determing root device (trying UUID=b840f739-b31f-4840-928f-a090aa59c90f) ...
>> Boot device detected as /dev/nvme1n1p2!
>> Mounting /dev/nvme1n1p2 as root ...
>> Using mount -t auto -o ro /dev/nvme1n1p2 /newroot
>> Switching to real root: /sbin/switch_root -c /dev/console /newroot /sbin/init
INIT: version 2.96 booting

    OpenRC 0.42.1 is starting up Gentoo Linux (x86_64)

/etc/fstab:
Code:
UUID=9F0A-C395                           /boot            vfat      defaults,noatime                           0 2
UUID=b840f739-b31f-4840-928f-a090aa59c90f      /               ext4      defaults,noatime                           0 1
tmpfs                                 /var/tmp/portage   tmpfs      uid=portage,gid=portage,mode=755,noatime         0 0

Kernel .config file: http://dpaste.com/35BM5Y4
lspci -k > http://dpaste.com/3PTBH23
dmesg > http://dpaste.com/0K9362K

No critical problems happen. I can boot my Gentoo, do something and shutdown it. No kernel panic occurs, but booting is slow. It takes a few seconds to try to load each modules which Genkernel will skip (nvme, sata, etc.). However, loading from virtio, hyperv and fs is immediate. Also, I've installed Gentoo on my laptop but although this Gentoo shows `skipping loading of module group foo` too, it boots fast and loads modules immediate.

Is it able to suppress these messages and increase booting speed?

(By the way, I typed messages from Genkernel because I didn't know where the logs were stored. dmesg didn't contain it and /var/log/genkernel.log includes the log of `genkernel --install initramfs`, not booting log. Where can I get the log?)


Last edited by toku on Fri Jan 17, 2020 11:06 am; edited 1 time in total
Back to top
View user's profile Send private message
sdauth
Tux's lil' helper
Tux's lil' helper


Joined: 19 Sep 2018
Posts: 91
Location: Frankreich

PostPosted: Fri Dec 27, 2019 10:41 am    Post subject: Reply with quote

Hi,

I have the same issue so I hope to get answers from this thread :o
One other thing I noticed is that when first generating the initramfs, genkernel 4.0.1 generates its own set of tools.
See :

/var/cache/genkernel/4.0.1/
Code:
LVM2.2.02.186-x86_64.tar.xz                  libaio-0.3.112-x86_64.tar.xz
boost-1.72.0-x86_64.tar.xz                   libgcrypt-1.8.5-x86_64.tar.xz
boost-build-1.72.0.tar.xz                    libgpg-error-1.36-x86_64.tar.xz
busybox-1.31.1-x86_64.tar.xz                 popt-1.16-x86_64.tar.xz
cryptsetup-2.2.2-x86_64.tar.xz               thin-provisioning-tools-0.8.5-x86_64.tar.xz
expat-2.2.8-x86_64.tar.xz                    util-linux-2.34-x86_64.tar.xz
json-c-0.13.1-x86_64.tar.xz


Before that, it used what was available on system, if it was compiled as static. In my case, cryptsetup, luks, busybox etc.. Why did it change ? I wonder.

Quote:
not booting log. Where can I get the log?

I enabled log in /etc/rc.conf

Code:
rc_logger="yes"
rc_log_path="/var/log/rc.log"


But it doesn't include the initramfs part. :?

Code:
rc sysinit logging started at Thu Dec 26 22:39:28 2019


   OpenRC 0.41.2 is starting up Gentoo Linux (x86_64)

[...]
Back to top
View user's profile Send private message
Whissi
Developer
Developer


Joined: 12 Jan 2011
Posts: 78

PostPosted: Thu Jan 02, 2020 8:03 pm    Post subject: Re: Too slow to load modules when booting Reply with quote

Hi,

toku wrote:
I noticed that when booting, Genkernel said "Root block device found, skipping loading of module group "foo" ...
Please read NOTE from https://wiki.gentoo.org/wiki/Genkernel: Genkernel will only load what's needed to get real system up and running. If you need additional modules you are responsible for loading them.

toku wrote:
No critical problems happen. I can boot my Gentoo, do something and shutdown it. No kernel panic occurs, but booting is slow. It takes a few seconds to try to load each modules which Genkernel will skip (nvme, sata, etc.). However, loading from virtio, hyperv and fs is immediate. Also, I've installed Gentoo on my laptop but although this Gentoo shows `skipping loading of module group foo` too, it boots fast and loads modules immediate.
Loading virtio/hyperv module group is so fast because in most cases these modules aren't present so there's nothing to load.

In theory, genkernel-4 should start faster than genkernel-3 due to "smart" module loading (was already present in genkernel-3 but for most users it wasn't working). "Smart" because we check for
Code:
root=
device after loading each module so we can end module loading as soon as possible, see https://gitweb.gentoo.org/proj/genkernel.git/tree/defaults/initrd.scripts?h=v4.0.1#n29. The "smart" behavior can be disabled by setting kernel command-line argument
Code:
gk.hw.load-all=yes
.

toku wrote:
By the way, I typed messages from Genkernel because I didn't know where the logs were stored. dmesg didn't contain it and /var/log/genkernel.log includes the log of `genkernel --install initramfs`, not booting log. Where can I get the log?)

Please set kernel command-line argument
Code:
gk.log.keep=yes
(see genkernel's man page or https://wiki.gentoo.org/wiki/Genkernel#Initramfs_kernel_command-line_parameters for more details). I am interested in this log file which will contain timestamps.

sdauth wrote:
Before that, it used what was available on system, if it was compiled as static. In my case, cryptsetup, luks, busybox etc.. Why did it change ? I wonder.
It's a bad idea to integrate arbitrary programs from host. And for proper cross-compile support it's required to build stuff genkernel uses on its own.

sdauth wrote:
I enabled log in /etc/rc.conf
Yes, see above. rc.log is from OpenRC.
_________________
Regards,
Whissi
Back to top
View user's profile Send private message
toku
n00b
n00b


Joined: 12 Jan 2019
Posts: 8

PostPosted: Fri Jan 03, 2020 2:41 am    Post subject: Reply with quote

Hi.
Quote:
Code:
gk.log.keep=yes

Thanks for telling me this. I added it to /etc/default/grub:
Code:
GRUB_CMDLINE_LINUX="gk.log.keep=yes"

genkernel logs:
with smart module loading:http://dpaste.com/01X29DF
without smart module loading:http://dpaste.com/0N5DS3M
Quote:
In theory, genkernel-4 should start faster than genkernel-3 due to "smart" module loading

To check this, I tried these things:
1. Add this line to /etc/portage/package.mask/mask
Code:
>=sys-kernel/genkernel-4.0.0

2. Delete all kernels and initramfses except the newest one.
Code:
sudo eclean-kernel

3. Install genkernel-3
Code:
sudo emerge -avt genkernel

Now:
Code:
genkernel --version
3.5.3.3

4. Run those.
Code:
sudo genkernel --install initramfs
sudo grub-mkconfig -o /boot/grub/grub.cfg

5. Reboot
Obviously genkernel-3 is faster than genkernel-4 with smart module loading.

I couldn't find the log in / even I set GRUB_CMDLINE_LINUX as this:
Code:
GRUB_CMDLINE_LINUX="gk.log.keep=yes"

Is this option introduced since version 4?
Back to top
View user's profile Send private message
Whissi
Developer
Developer


Joined: 12 Jan 2011
Posts: 78

PostPosted: Fri Jan 03, 2020 12:51 pm    Post subject: Reply with quote

Yes, the log option is new in genkernel-4.

You don't need to downgrade to genkernel-4. To disable "smart" loading, set kernel command-line argument
Code:
gk.hw.load-all=yes


Can you tell me something about your system? What's your root= value? Where is root= located on? Is this a very slow system in general? Do you have many disks to enumerate? At the moment it looks like findfs() takes 2s on your system. So loading 11-12 groups will take at least ~24s.
_________________
Regards,
Whissi
Back to top
View user's profile Send private message
toku
n00b
n00b


Joined: 12 Jan 2019
Posts: 8

PostPosted: Fri Jan 03, 2020 3:05 pm    Post subject: Reply with quote

Thanks for replying.

First of all, I noticed that booting became slow when CD-ROM was inserted. If it's not, loading each modules (and skipping it) doesn't take time.
Quote:
Can you tell me something about your system? What's your root= value? Where is root= located on?

From /boot/grub/grub.cfg
Code:
linux   /vmlinuz-5.4.7-gentoo root=UUID=b840f739-b31f-4840-928f-a090aa59c90f ro gk.log.keep=yes

Actually this UUID is root's one (/) . See my /etc/fstab which is written in the first post of this thread.
Full /boot/grub/grub.cfg:http://dpaste.com/18GBNE4
Quote:
Is this a very slow system in general?

Never. My computer equips i9-9900K CPU which should be fast.
Quote:
Do you have many disks to enumerate?

I installed Gentoo on only one SSD.

But why booting with smart loading becomes slow when CD is inserted? When smart loading is off, module loading process is immediate even CD is inserted.
Back to top
View user's profile Send private message
Whissi
Developer
Developer


Joined: 12 Jan 2011
Posts: 78

PostPosted: Fri Jan 03, 2020 3:28 pm    Post subject: Reply with quote

Heh, now I need to find a real system with optical drive to test? :-)

I guess findfs will also enumerate optical drives.

I think I'll add a marker once rootfs was found. This should skip additional findfs calls.
_________________
Regards,
Whissi
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 14971

PostPosted: Sat Jan 04, 2020 6:12 pm    Post subject: Reply with quote

Whissi wrote:
Heh, now I need to find a real system with optical drive to test? :-)
You might be able to use the device mapper's delay target to create an artificially slow device for findfs to search. See Documentation/admin-guide/device-mapper/delay.rst for minimal usage instructions. This probably requires CONFIG_DM_DELAY=y (or =m) in your kernel configuration.
Back to top
View user's profile Send private message
Whissi
Developer
Developer


Joined: 12 Jan 2011
Posts: 78

PostPosted: Sun Jan 05, 2020 9:27 pm    Post subject: Reply with quote

OK, I was able to reproduce (thanks Hu but I found an optical device): https://gitweb.gentoo.org/proj/genkernel.git/commit/?id=ea892515ef3a46171d288d9c119defedf8dbee28

Will be in genkernel-4.0.2.
_________________
Regards,
Whissi
Back to top
View user's profile Send private message
toku
n00b
n00b


Joined: 12 Jan 2019
Posts: 8

PostPosted: Sat Jan 11, 2020 3:01 pm    Post subject: Reply with quote

Sorry for not replying to your post. I didn't notice it.
And thank you for dealing with this problem. I'll check it out when I emerge genkernel-4.0.2. (currently the latest version is 4.0.1)
Back to top
View user's profile Send private message
toku
n00b
n00b


Joined: 12 Jan 2019
Posts: 8

PostPosted: Fri Jan 17, 2020 11:06 am    Post subject: Reply with quote

Loading modules with CD inserted became faster! Thank you @Whissi !
Back to top
View user's profile Send private message
Whissi
Developer
Developer


Joined: 12 Jan 2011
Posts: 78

PostPosted: Fri Jan 17, 2020 1:21 pm    Post subject: Reply with quote

Thank you for the feedback!
_________________
Regards,
Whissi
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum