grub "root=ZFS=..." misses zpool

[heilts]

Hello,

I use Debian "Jessie" with zfs 0.6.4-1.1-2 and root on zfs. The root pool is "tank0".

After upgrading to grub 2.02-beta2.9-ZOL1, the grub.cfg misses the pool part, i.e. it becomes e.g.

linux /jessie-1/ROOT@/boot/vmlinuz-3.19.8 root=ZFS=jessie-1/ROOT ro boot=zfs $bootfs quiet

This does not work, it should be e.g.

linux /jessie-1/ROOT@/boot/vmlinuz-3.19.8 root=ZFS=tank0/jessie-1/ROOT ro boot=zfs $bootfs quiet

Also, at running "update-grub", I got

root@tor:~# update-grub
Generating grub configuration file ...
Found background image: /usr/share/images/desktop-base/desktop-grub.png
Found linux image: /boot/vmlinuz-3.19.8
Found initrd image: /boot/initrd.img-3.19.8
/usr/sbin/grub-probe: error: unknown filesystem.
Found linux image: /boot/vmlinuz-3.19.3
Found initrd image: /boot/initrd.img-3.19.3
Found linux image: /boot/vmlinuz-3.16.0-4-amd64
Found initrd image: /boot/initrd.img-3.16.0-4-amd64
/usr/sbin/grub-probe: error: unknown filesystem.
Found memtest86+ image: /jessie-1/ROOT@/boot/memtest86+.bin
Found memtest86+ multiboot image: /jessie-1/ROOT@/boot/memtest86+_multiboot.bin
Found Debian GNU/Linux (8.0) on /dev/sdi1
done

The root filesystem is placed on GPT parted disks (raidz1) partitition 4. The other partititions are used for gtp grub and md1 (carrying an ext4 based rescue system and swap, this is why the update-grub creates so many entries). In more detail:
3.19.8 and 3.19.3 are one zfs (active root) only.
3.16.0-4-amd64 andn memtest are one zfs and ext4 (rescue).
And there is a complete backup of zfs placed on /dev/sdi1. This looks to break GRUB as well, because it creates only entries for 3.16.0-4-amd64, these multiple times, with increasing numbers of auf /dev/sdi1, looking like:

        menuentry 'Debian GNU/Linux (auf /dev/sdi1) (auf /dev/sdi1) (auf /dev/sdi1) (auf /dev/sdi1) (auf /dev/sdi1) (auf /dev/sdi1) (auf /dev/sdi1) (on /dev/sdi1)' --class gnu-linux --class gnu --class os $menuentry_id_option 'osprober-gnulinux-/boot/vmlinuz-3.16.0-4-amd
64--51a2e90e-b2a5-4006-ba1b-db85588a6881' {
                insmod part_msdos
                insmod ext2
                set root='hd7,msdos1'
                if [ x$feature_platform_search_hint = xy ]; then
                  search --no-floppy --fs-uuid --set=root --hint-bios=hd8,msdos1 --hint-efi=hd8,msdos1 --hint-baremetal=ahci8,msdos1 --hint='hd7,msdos1'  51a2e90e-b2a5-4006-ba1b-db85588a6881
                else
                  search --no-floppy --fs-uuid --set=root 51a2e90e-b2a5-4006-ba1b-db85588a6881
                fi
                linux /boot/vmlinuz-3.16.0-4-amd64 root=UUID=8e3538db-de20-4ef4-81f2-9464fdeed165 ro quiet
                initrd /boot/initrd.img-3.16.0-4-amd64
        }

grub-probe works:

root@tor:~# grub-probe /
zfs

GRUB installation is:

root@tor:~# dpkg -l \*grub\* | grep ^ii
ii  grub-common     2.02-beta2.9-ZOL11-7aa9f6 amd64        GRand Unified Bootloader (common files)
ii  grub-pc         2.02-beta2.9-ZOL11-7aa9f6 amd64        GRand Unified Bootloader, version 2 (PC/BIOS version)
ii  grub-pc-bin     2.02-beta2.9-ZOL11-7aa9f6 amd64        GRand Unified Bootloader, version 2 (PC/BIOS binaries)
ii  grub2-common    2.02-beta2.9-ZOL11-7aa9f6 amd64        GRand Unified Bootloader (common files for version 2)

[heilts]

I tracked a bit further down:
In 10_linux the command line is build:

10_linux:       rpool=`${grub_probe} --device ${GRUB_DEVICE} --target=fs_label 2>/dev/null || true`
10_linux:       LINUX_ROOT_DEVICE="ZFS=${rpool}${bootfs}"

But rpool becomes empty, because the grub_probe fails:

root@tor:/etc/grub.d# /usr/sbin/grub-probe --device /dev/sdb4 --target=fs_label
/usr/sbin/grub-probe: error: unknown filesystem.

Same happens with sdba4.

Note: My zfs root is placed on sda4 and sdb4 in raidz1.

[dechamps]

I got the exact same issue on Debian Sid, grub-common 2.02-beta2.9-ZOL11-7aa9f6. Using /, grub-probe works:

# grub-probe --target device /
/dev/sdd1
/dev/sdg1
/dev/sda1
/dev/sde1
/dev/sdc1
/dev/sdf1

However, when using grub-probe with a device node (which is what grub-mkconfig and 10_linux do), it doesn't:

# grub-probe --target fs --device /dev/sdd1
grub-probe: error: unknown filesystem.
$ sudo grub-probe --target fs_label --device /dev/sdd1
grub-probe: error: unknown filesystem.

I pasted -vvv output into https://gist.github.com/dechamps/8ebdf9c3631bd0cda2ef.

[dechamps]

For now, this can be worked around without touching package files by manually specifying root=ZFS=... in /etc/default/grub, in the GRUB_CMDLINE_LINUX variable. The contents of this variable are appended after the generated root stanza, which means the broken root will be overriden.

[firaxis]

I'm trying to install debian with zfs on root.
It installs and works with single vdev in pool or mirror, but not with raidz as boot pool.

[Spongman]

is this repo still active? i'm seeing exactly the same issue, 18 months later...

[Spongman]

apparently the bug is due to the fact that grub-mkconfig is doing a 'head -n1' on your grub-probe output. and later on when grub-probe is called again it needs to be passed a quorum of devices in order to determine the filesystem type and label.

[firaxis]

grub-probe can't determine filesystem type when you are trying to get it from one of raidz disks.
Also it may depend on array size, e.g. grub-probe can't do it on array of > 6 disks.

[pstch]

This can be worked around by editing /etc/grub.d/10_linux and replacing:

rpool=`${grub_probe} --device ${GRUB_DEVICE} --target=fs_label 2>/dev/null || true`

By :

rpool=`zdb -l ${GRUB_DEVICE} | grep " name:" | cut -d\' -f2`

You can also just hardcode your root pool name:

rpool="rpool"

[exikyut]

Hello everyone, I believe I've just crashed into exactly this issue, an extremely similar issue, or the same outcome caused by different circumstances.

In my case, grub-probe is not detecting the root pool due to unsupported features. It is incidentally actually able to detect an auxillary pool though:

# grub-probe -t fs_label -d /dev/vda3
grub-probe: error: unknown filesystem.
# grub-probe -t fs_label -d /dev/vda4
pool-1

Adding -vvvv reveals that it bails out during feature detection:

grub-core/fs/zfs/zfs.c:2115: zap: name = org.illumos:lz4_compress, value = 1, cd = 0
grub-core/fs/zfs/zfs.c:2115: zap: name = com.joyent:multi_vdev_crash_dump, value = 0, cd = 0
grub-core/fs/zfs/zfs.c:2115: zap: name = com.delphix:hole_birth, value = 1, cd = 0
grub-core/fs/zfs/zfs.c:2115: zap: name = com.delphix:extensible_dataset, value = 1, cd = 0
grub-core/fs/zfs/zfs.c:2115: zap: name = com.delphix:embedded_data, value = 1, cd = 0
grub-core/fs/zfs/zfs.c:2115: zap: name = org.open-zfs:large_blocks, value = 0, cd = 0
grub-core/fs/zfs/zfs.c:2115: zap: name = org.zfsonlinux:large_dnode, value = 1, cd = 0
grub-core/kern/fs.c:78: zfs detection failed.

versus

grub-core/fs/zfs/zfs.c:2115: zap: name = org.illumos:lz4_compress, value = 1, cd = 0
grub-core/fs/zfs/zfs.c:2115: zap: name = com.joyent:multi_vdev_crash_dump, value = 0, cd = 0
grub-core/fs/zfs/zfs.c:2115: zap: name = com.delphix:hole_birth, value = 1, cd = 0
grub-core/fs/zfs/zfs.c:2115: zap: name = com.delphix:extensible_dataset, value = 0, cd = 0
grub-core/fs/zfs/zfs.c:2115: zap: name = com.delphix:embedded_data, value = 1, cd = 0
grub-core/fs/zfs/zfs.c:2115: zap: name = org.open-zfs:large_blocks, value = 0, cd = 0
grub-core/fs/zfs/zfs.c:2115: zap: name = org.zfsonlinux:large_dnode, value = 0, cd = 0
grub-core/fs/zfs/zfs.c:2115: zap: name = org.illumos:sha512, value = 0, cd = 0
grub-core/fs/zfs/zfs.c:2115: zap: name = org.illumos:skein, value = 0, cd = 0
grub-core/fs/zfs/zfs.c:2115: zap: name = org.illumos:edonr, value = 0, cd = 0
grub-core/fs/zfs/zfs.c:2115: zap: name = com.datto:bookmark_v2, value = 0, cd = 0
grub-core/fs/zfs/zfs.c:2115: zap: name = com.datto:encryption, value = 0, cd = 0
grub-core/fs/zfs/zfs.c:2115: zap: name = com.delphix:device_removal, value = 0, cd = 0
grub-core/fs/zfs/zfs.c:2115: zap: name = , value = 0, cd = 0
grub-core/fs/zfs/zfs.c:2115: zap: name = , value = 0, cd = 0
pool-1
grub-core/kern/disk.c:295: Closing `hostdisk//dev/vda'.

Note how large_dnode is 1 in the failing case; this value is not in the whitelist.

I make the (IMO reasonably solid) argument over at https://savannah.gnu.org/bugs/index.php?58555 that this is because grub-probe is brokenly being used to analyze root filesystems with features enabled that it does not support.

If I remember :) I'll update this thread.

[jstrieter]

This can be worked around by editing /etc/grub.d/10_linux and replacing:

rpool=`${grub_probe} --device ${GRUB_DEVICE} --target=fs_label 2>/dev/null || true`

By :

rpool=`zdb -l ${GRUB_DEVICE} | grep " name:" | cut -d\' -f2`

You can also just hardcode your root pool name:

rpool="rpool"

This workaround fixes it but it's still an issue as of 2.05_alpha20200310.

[crocket]

This is an issue on my machine with grub-2.05_alpha20200310

[MilesBHuff]

What's the long wait on this issue for? It's been open for 6 years. There's even a fix in the comments.