Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[solved]fail to boot ZFS root after update to 5.4.28
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
spofforth
n00b
n00b


Joined: 16 Nov 2019
Posts: 4

PostPosted: Fri May 01, 2020 10:45 pm    Post subject: [solved]fail to boot ZFS root after update to 5.4.28 Reply with quote

Initial configuration was done using an Ubuntu liveCD, genkernel and 4.19.82. I used a few guides as... guidance: fearedbliss, funtoo and guyrobot.

It's working, i'm using it now. But i'd like to upgrade the kernel and lose unnecessary drivers.
Hardware's an ASUS Prime X370-A with a Ryzen 7 1800X.
The ZFS pool is a 3-disk RAIDZ. No other partiitons.

Take 1

Got current config, upgraded to new kernel's config with default values, removed unused modules, screwed around with menuconfig.

Code:
zcat /proc/config.gz > .config

Code:
make olddefconfig

Got this warning: .config:1069:warning: symbol value 'm' invalid for NF_CT_PROTO_GR
Code:
make localmodconfig

Had to manually set Studio Evolution SE6X (SND_SE6X) [N/m/?] (NEW)
Code:
make menuconfig

Using the handbook, Ryzen and ZFS wiki pages for reference.

Then generated the initramfs:
Code:
genkernel --no-clean --no-mountboot --makeopts=-j8 --kernel-config=/usr/src/linux-5.4.28-gentoo/.config --keymap --microcode=amd --microcode-initramfs --bootloader=grub2 --real-root=ZFS=tank/ROOT/gentoo --zfs --install initramfs


This was the result:
Code:
   ls -ltr /boot
   ...
   -rw-r--r-- 1 root root    31744 Apr 26 01:04 amd-uc.img
   -rw-r--r-- 1 root root  7857536 Apr 26 11:22 vmlinuz-5.4.28-gentoo
   -rw-r--r-- 1 root root  3519746 Apr 26 11:22 System.map-5.4.28-gentoo
   -rw-r--r-- 1 root root   131142 Apr 26 11:22 config-5.4.28-gentoo
   -rw-r--r-- 1 root root 16848312 Apr 26 23:28 initramfs-5.4.28-gentoo.img


However, it fails to boot, saying:
Code:
Block device ZFS=tank/ROOT/gentoo is not a valid root device.
Could not find the root block device in ZFS=tank/ROOT/gentoo


So i assumed either i hadn't included ZFS modules in the ramdisk or i left some driver out when screwing around with the kernel.

Take 2

Elsewhere in this forum found some leads to use genkernel for everything. Since the current kernel is working, i thought i'd try just to upgrade it to the latest without further tweaking:
Code:
genkernel --no-mountboot --makeopts=-j8 --kernel-config=/proc/config.gz --keymap --microcode=amd --microcode-initramfs --bootloader=grub2 --real-root=ZFS=tank/ROOT/gentoo --zfs --install all


While generating initramfs i get this warning:
Code:
      *         >> Appending zfs cpio data ...
      *                 zfs: /etc/hostid not found; You must use 'spl_hostid' kernel command-line parameter!

Searching around, this shouldn't be an issue. I think it's due to the fact that i installed using a system (the liveCD) other than the system that currently uses the pool. This is also the reason zpool status gives me this warning:
Code:
status: Mismatch between pool hostid and system hostid on imported pool.
   This pool was previously imported into a system with a different hostid,
   and then was verbatim imported into this system.
action: Export this pool on all systems on which it is imported.
   Then import it to correct the mismatch.
   see: http://zfsonlinux.org/msg/ZFS-8000-EY

It hasn't prevented the system from booting since i installed it though. I could either
  • run hostid > /etc/hostid
  • set spl_hostid=0xTheId
  • set spl_hostid=0 (i believe it's the default
  • echo 0 > /etc/hostid
Changing /etc/hostid alone would mean re-creating the initramfs. I did neither at the moment. However, the message ends with
Code:
      * No hostid embedded into initramfs. You MUST set 'spl_hostid=<hostid>' parameter to provide hostid for ZFS!


There's also the other warning:
Code:
      * WARNING... WARNING... WARNING...
      * Additional kernel parameters that *may* be required to boot properly:
      * - Add "dozfs" for ZFS volume management support
      *   and either "root=ZFS" to use bootfs autodetection or "root=ZFS=<dataset>"
      *   to force booting from a specific dataset

but this is already included.

This seemed to advance a bit, but not quite. I now get
Code:
importing ZFS pool tank/ROOT/gentoo
tank/ROOT/gentoo is not a filesystem


Bootloader

On both runs i re-generated grub.cfg, i haven't touched it really.
Code:
grub-mkconfig -o /boot/grub/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/kernel-genkernel-x86_64-4.19.82-gentoo
Found initrd image: /boot/amd-uc.img /boot/initramfs-genkernel-x86_64-4.19.82-gentoo
Found linux image: /boot/vmlinuz-5.4.28-gentoo-x86_64
Found initrd image: /boot/amd-uc.img /boot/initramfs-5.4.28-gentoo-x86_64.img
done


My /etc/default/grub
Code:
GRUB_CMDLINE_LINUX="dozfs real_root=ZFS=tank/ROOT/gentoo"
GRUB_CMDLINE_LINUX_DEFAULT="dozfs real_root=ZFS=tank/ROOT/gentoo"


Code:
grub-probe /
zfs


This is the current kernel line in grub.cfg:
Code:
        menuentry 'Gentoo GNU/Linux, with Linux x86_64-4.19.82-gentoo'
  ...
        echo    'Loading Linux x86_64-4.19.82-gentoo ...'
        linux   /ROOT/gentoo@/boot/kernel-genkernel-x86_64-4.19.82-gentoo root=ZFS=tank/ROOT/gentoo ro dozfs real_root=ZFS=tank/ROOT/gentoo dozfs real_root=ZFS=tank/ROOT/gentoo
        echo    'Loading initial ramdisk ...'
        initrd  /ROOT/gentoo@/boot/amd-uc.img /ROOT/gentoo@/boot/initramfs-genkernel-x86_64-4.19.82-gentoo

And the new one:
Code:
        menuentry 'Gentoo GNU/Linux, with Linux 5.4.28-gentoo-x86_64'
  ...
                echo    'Loading Linux 5.4.28-gentoo-x86_64 ...'
                linux   /ROOT/gentoo@/boot/vmlinuz-5.4.28-gentoo-x86_64 root=ZFS=gpool/ROOT/gentoo ro dozfs real_root=ZFS=gpool/ROOT/gentoo dozfs real_root=ZFS=gpool/ROOT/gentoo
                echo    'Loading initial ramdisk ...'
                initrd  /ROOT/gentoo@/boot/amd-uc.img /ROOT/gentoo@/boot/initramfs-5.4.28-gentoo-x86_64.img

They both do insmod zfs. I spot no differences.

I did notice this:
Code:
# file /boot/initram*
/boot/initramfs-5.4.28-gentoo-x86_64.img:         ASCII cpio archive (SVR4 with no CRC)
/boot/initramfs-genkernel-x86_64-4.19.82-gentoo:  XZ compressed data


At the moment these are the versions i have:
sys-fs/zfs 0.8.3-r1
sys-fs/zfs-kmod 0.8.3

I thought about running grub-install but i don't think that'd be the issue as it correctly finds the kernels and the ramdisks already so the bootloader proper doesn't need changing. But i'm confused as to what did i miss.
Any hints?

Files:
https://pastebin.fun/50UrctpfTs /etc/default/grub
https://pastebin.fun/Md0GAkEcxf /boot/grub/grub.cfg
https://pastebin.fun/Ok1xhaiGTD 4.19.82 .config (working)
https://pastebin.fun/pG19l4oTvD 5.4.28 .config
Back to top
View user's profile Send private message
mrbassie
Guru
Guru


Joined: 31 May 2013
Posts: 592

PostPosted: Sun May 03, 2020 8:33 pm    Post subject: Reply with quote

Did you remember to do
Code:
emerge @module-rebuild


after upgrading the kernel?
Back to top
View user's profile Send private message
spofforth
n00b
n00b


Joined: 16 Nov 2019
Posts: 4

PostPosted: Tue May 05, 2020 8:07 pm    Post subject: Reply with quote

mrbassie wrote:
Did you remember to do
Code:
emerge @module-rebuild


after upgrading the kernel?


Probably not. I've noticed you can also use -callback="emerge @module-rebuild" when calling genkernel.
Even so, same result.

I dropped to a shell and zpool status gave me "no pools available", which is kind of odd.

I'm thinking there may be something different when defining the root filesystem, maybe ZFS=tank instead of ZFS=tank/ROOT/gentoo but it works with 4.19.82.
Or maybe i'll need to change to /dev/sda2 format or something.

For the sake of completeness this is the ZFS config:
Code:
# ll /dev/disk/by-label/
rwxrwxrwx 1 root root 10 May  5 21:08 tank -> ../../sda1

# ll /dev/disk/by-partlabel/
lrwxrwxrwx 1 root root 10 May  5 17:32  zfs-063d83ec3506186a -> ../../sdc1
lrwxrwxrwx 1 root root 10 May  5 17:32  zfs-3a2ea0addf84a90c -> ../../sdb1
lrwxrwxrwx 1 root root 10 May  5 21:08  zfs-b65f06ed732ffede -> ../../sda1

# zpool status
   NAME        STATE     READ WRITE CKSUM
   tank        ONLINE       0     0     0
     raidz1-0  ONLINE       0     0     0
       sdb     ONLINE       0     0     0
       sda     ONLINE       0     0     0
       sdc     ONLINE       0     0     0

# zfs list
NAME                USED  AVAIL     REFER  MOUNTPOINT
tank                296G  1.47T      128K  none
tank/ROOT           296G  1.47T      128K  none
tank/ROOT/gentoo   17.0G  1.47T     14.2G  /
tank/ROOT/home     71.1G  1.47T     71.1G  /home
tank/ROOT/srv       208G  1.47T      208G  /srv


I'll have to check it out once i drop into a shell again, and see if zpool import -a does anything.
Back to top
View user's profile Send private message
spofforth
n00b
n00b


Joined: 16 Nov 2019
Posts: 4

PostPosted: Wed May 06, 2020 5:57 pm    Post subject: Reply with quote

Actually... hostid is indeed mandatory (now). I thought it wasn't, according to Arch.

So,
Code:
rm /etc/hostid # if it exists
zgenhostid $(hostid) # so it uses the same value as generated by hostid and place it into /etc/hostid

Then add spl.spl_hostid=0xdeadbeef to GRUB_CMDLINE_LINUX= in /etc/default/grub, "deadbeef" being whatever 8 hex characters hostid spews (cat-ing /etc/hostid might produce "garbage"... well it's in hex).
(Then you'll need to run grub-mkconfig -o /boot/grub/grub.cfg, or edit /boot/grub/grub.cfg (prefer the former).

You can troubleshoot this with
Code:
shell # drop to a shell on the boot when it fails and misleadingly tells you tank/ROOT/gentoo it's not a fielsystem
zfs status # will tell you there are no pools available, which is false
zfs import -a # will tell you it failed to import tank 'cos it was mounted on another system with hostid 0xdeadbeef (corresponding to your hostid actually)
zfs import -f tank # will indeed import it, and Ctrl+D-ing will leave the shell and boot


(PS i'd add "solved" but can't edit the original post's subject?)
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 15294

PostPosted: Thu May 07, 2020 12:57 am    Post subject: Reply with quote

spofforth wrote:
(PS i'd add "solved" but can't edit the original post's subject?)
The subject was too long. I shortened it to make the marker fit.
Back to top
View user's profile Send private message
ununu
n00b
n00b


Joined: 19 Apr 2020
Posts: 29

PostPosted: Mon May 11, 2020 3:35 pm    Post subject: Reply with quote

spofforth wrote:
Actually... hostid is indeed mandatory (now). I thought it wasn't, according to Arch


greets,

here, on a test setup, booting root zfs without hostid. So it's not that zfs can't find itself without it or any other type of boot loader and zfs combinations.

Well, reading man zpool hostid got to do with multihost=on|off option. In turn it might be related to having zfs-share and zfs-zed on some of your runlevels.

I hope this helps other people trying to troubleshoot a gentoo zfs root install.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum