Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[SOLVED] unable to mount rootfs after upgrading to 4.19.44
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
arcctgx
n00b
n00b


Joined: 07 Jul 2014
Posts: 24

PostPosted: Sat Jun 01, 2019 6:27 pm    Post subject: [SOLVED] unable to mount rootfs after upgrading to 4.19.44 Reply with quote

Hi everyone,

Today I have updated the kernel from 4.19.27-gentoo-r1 to 4.19.44-gentoo without any configuration changes. With the new kernel I have a booting problem if an USB device (pendrive or similar) is plugged in when the system is booting. (If my Kindle wasn't plugged in for charging I'd never have noticed).

It looks as if there is a race condition. Sometimes the pendrive gets detected before the hard drive and gets assigned /dev/sda1, which causes "Kernel panic, unable to mount rootfs" jazz. I tried rebooting many times with a pendrive plugged in using both old and new kernel. With 4.19.44 the pendrive is detected first around 3 out of 4 times. But it never happens with 4.19.27, the hard disk is always detected first.

Initially I thought that the solution would be to change my /etc/fstab to use UUIDs instead of specifying the devices "the old way" as /dev/sdXY. So I did that, but this did not help: the system boots fine with 4.19.44 without any pendrives plugged in, but I'm still getting the behavior described above when the USB device is connected.

I'm probably missing something simple. Perhaps there is some kernel configuration option that I need to enable with the new kernel? Or something else entirely? I would appreciate your help.


Last edited by arcctgx on Sat Jun 01, 2019 9:44 pm; edited 1 time in total
Back to top
View user's profile Send private message
ct85711
Veteran
Veteran


Joined: 27 Sep 2005
Posts: 1741

PostPosted: Sat Jun 01, 2019 7:00 pm    Post subject: Reply with quote

The part you need, to use is PARTUUID not UUID. To use UUID, you need to use a initrd (to read the filesystem), where PARTUUID (partition level) is understood by the kernel without anything, so no initrd is needed.

Included is a link to a previous thread on the same issue (this issue is not dependent on kernel version).
https://forums.gentoo.org/viewtopic-t-1079474-start-0.html
Back to top
View user's profile Send private message
arcctgx
n00b
n00b


Joined: 07 Jul 2014
Posts: 24

PostPosted: Sat Jun 01, 2019 7:43 pm    Post subject: Reply with quote

Thanks for your input, but unfortunaltely using PARTUUIDs instead of UUIDs im my /etc/fstab did not solve the issue I'm having.

I read the thread you were referring to, but I don't see how it applies to my case. In my case kernel correctly detects the rootfs UNLESS there is a USB mass storage device plugged in at boot time.

Some additional info that may be relevant: I'm using grub-2.02-r3, without an initrd. I'm not using disk encryption.

Right now it occured to me that some additional bootloader configuration may be necessary. In my grub.cfg (generated automatically with grub-mkconfig) I can see the following line:
Code:
linux   /vmlinuz-4.19.44-gentoo root=/dev/sda4 ro printk.time=0
For some reason grub-mkconfig is still using /dev/sdXY notation for root. This file is not supposed to be edited manually, so I probably need to change something in /etc/default/grub.

The funny thing is that the manual of grub says:
Quote:
‘GRUB_DISABLE_LINUX_UUID’

Normally, grub-mkconfig will generate menu entries that use universally-unique identifiers (UUIDs) to identify the root filesystem to the Linux kernel, using a ‘root=UUID=...’ kernel parameter. This is usually more reliable, but in some cases it may not be appropriate. To disable the use of UUIDs, set this option to ‘true’.
I had this option commented out by default. Later I set this option explicitly to "false", but grub-mkconfig is still using /dev/sdXY instead of a UUID.

How do I correctly pass the UUID of my root partition to grub?
Back to top
View user's profile Send private message
Jaglover
Watchman
Watchman


Joined: 29 May 2005
Posts: 7506
Location: Saint Amant, Acadiana

PostPosted: Sat Jun 01, 2019 7:55 pm    Post subject: Reply with quote

Quote:
unfortunaltely using PARTUUIDs instead of UUIDs im my /etc/fstab did not solve the issue I'm having.


Your fstab is residing in root filesystem, how could your kernel possibly read it if it cannot mount the root filesystem?
_________________
Please learn how to denote units correctly!
Back to top
View user's profile Send private message
arcctgx
n00b
n00b


Joined: 07 Jul 2014
Posts: 24

PostPosted: Sat Jun 01, 2019 8:22 pm    Post subject: Reply with quote

Argh, of course. So editing fstab will not help at all. Now I feel dumb. :(

So, what should I do to avoid the problem I described? Any idea? (other that not plugging in USB mass storage devices at boot time?) ;)

And why does it happen with kernel 4.19.44 but not with 4.19.27?
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 14971

PostPosted: Sat Jun 01, 2019 9:09 pm    Post subject: Reply with quote

Perhaps the older kernel does not understand that form of USB mass storage, or is configured in a way that it detects it much later, such as after modules load.

For the fix, you should make grub pass an appropriate root=PARTUUID= to your kernel.
Back to top
View user's profile Send private message
tomtom69
Apprentice
Apprentice


Joined: 09 Nov 2010
Posts: 205
Location: Bavaria

PostPosted: Sat Jun 01, 2019 9:34 pm    Post subject: Reply with quote

I also had this kind of problem some time ago. It really looks like a race condition which causes the drive enumeration to change dependent on the kernel version if an USB drive is connected in addition to the (internal) SATA drives.
My remedy was to compile the USB storage driver as a module. This causes the USB drives to be always enumerated after rootfs mount which fixed the problem permanently. Might not be "elegant" but did the job.

tom
Back to top
View user's profile Send private message
arcctgx
n00b
n00b


Joined: 07 Jul 2014
Posts: 24

PostPosted: Sat Jun 01, 2019 9:43 pm    Post subject: Reply with quote

OK, so I think I got it right this time. The solution is to pass the correct PARTUUID to the kernel, AND to use PARTUUID in /etc/fstab. That way it doesn't matter if the hard drive is detected as /dev/sda or /dev/sdb or whatever else.

Passing PARTUUID to the kernel is the crucial thing. I did that by adding root=PARTUUID=xxxxxxxx-yyyy-zzzz-aaaa-bbbbbbbbbbbb to GRUB_CMDLINE_LINUX variable in my /etc/defaults/grub. But it seems to be something that grub should figure out by itself, and I have no idea why it's unable to do so. Furthermore, it generates the following grub.cfg content:
Code:
linux   /vmlinuz-4.19.44-gentoo root=/dev/sda4 ro root=PARTUUID=xxxxxxxx-yyyy-zzzz-aaaa-bbbbbbbbbbbb
The ugly thing is that root= is specified twice, but fortunately the latter root= option seems to take precedence over the former.

Thanks everyone, it was a good discussion which pointed me in the right direction.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 44945
Location: 56N 3W

PostPosted: Sat Jun 01, 2019 9:45 pm    Post subject: Reply with quote

arcctgx,

To generalise the advice already provided, you need to avoid using /dev nodes to describe your filesystem to the kernel at all times.
/dev/sda and friends are allocated on a first come first named basis.
When USB storage devices are in the mix, the order can vary from boot to boot never mind kernel to kernel.

In /etc/fstab use UUID, PARTUUID or LABEL. It matters for getting started as the root entry is consulted to know what filesystem type root is for rootfsck.
As has already been said, it cannot be used to locate the root filesystem itself.

LABEL and UUID, need the userspace mount program, so to use then on the kernel command line, an initrd is required as they are filesystem properties.
The kernel is quite happy with PARTUUID its a property of a partition, so use PARTUUID on the kernel command line.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
arcctgx
n00b
n00b


Joined: 07 Jul 2014
Posts: 24

PostPosted: Sat Jun 01, 2019 10:47 pm    Post subject: Reply with quote

What you just wrote should be added to https://wiki.gentoo.org/wiki/Fstab. :)

Edit:
I just had to update the configuration of hddtemp to use appropriate symlink from /dev/disk/by-id/ instead of /dev/sdXY...
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 44945
Location: 56N 3W

PostPosted: Sun Jun 02, 2019 10:56 am    Post subject: Reply with quote

arcctgx,

Using /dev/disk/by-* symlinks is dangerous. It exposes you to a race condition.

The symlinks are created by udev at boot time.
If you try to use them before udev has created them, you get to keep the pieces.

Its probably ok for hddtemp, since if it misses a reading, you probably won't notice.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
arcctgx
n00b
n00b


Joined: 07 Jul 2014
Posts: 24

PostPosted: Sun Jun 02, 2019 12:08 pm    Post subject: Reply with quote

Quote:
Using /dev/disk/by-* symlinks is dangerous. It exposes you to a race condition
I figured there is a possibility of a race condition, though this solution seems good enough for hddtemp. If I'm not supposed to use /dev/sdXY or any of the symlinks created by udev in /dev/disk/, what other option do I have?

One more question if I may. There is an option CONFIG_PM_STD_PARTITION in the kernel configuration. I suppose I should pass resume=PARTUUID=... parameter to the kernel via grub, instead of configuring the resume partition in the kernel?
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 14971

PostPosted: Sun Jun 02, 2019 4:38 pm    Post subject: Reply with quote

As I read kernel/power/hibernate.c, setting CONFIG_PM_STD_PARTITION assigns an initial value to resume_file. Passing resume= assigns to resume_file at runtime. Therefore, setting CONFIG_PM_STD_PARTITION="PARTUUID=" and passing resume=PARTUUID= have the same effect. You may do whichever you prefer, and can use the runtime value to override one built in to the kernel.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum