ZTS: zvol/zvol_misc/zvol_misc_volmode failures

[behlendorf]

System information

Type	Version/Name
Distribution Name	FreeBSD
Distribution Version	13,14
Kernel Version
Architecture	x86_64
OpenZFS Version	2.3-rc2

Describe the problem you're observing

On FreeBSD the zvol/zvol_misc/zvol_misc_volmode test occasionally fails unexpectedly.

Describe how to reproduce the problem

Can be reproduced occasionally by the CI.

https://github.com/openzfs/zfs/actions/runs/11420934502/job/31777190630?pr=16625

Include any warning/errors/backtraces from the system logs

Test: /usr/local/share/zfs/zfs-tests/tests/functional/zvol/zvol_misc/zvol_misc_volmode (run as root) [00:16] [FAIL]
  02:13:24.03 ASSERTION: Verify that ZFS volume property 'volmode' works as intended
  02:13:24.06 SUCCESS: zfs create -o mountpoint=none testpool/volfs
  02:13:24.09 SUCCESS: zfs create -V 256m -s testpool/volfs/subvol
  02:13:24.16 SUCCESS: zfs create -V 256m -s testpool/vol
  02:13:24.16 SUCCESS: dd if=/dev/zvol/testpool/vol of=/dev/null count=1
  02:13:24.21 SUCCESS: dd if=/dev/zero of=/dev/zvol/testpool/vol count=1
  02:13:24.21 SUCCESS: dd if=/dev/zvol/testpool/volfs/subvol of=/dev/null count=1
  02:13:24.25 SUCCESS: dd if=/dev/zero of=/dev/zvol/testpool/volfs/subvol count=1
  02:13:24.26 SUCCESS: zfs set volmode=off testpool/vol exited 255
  02:13:24.26 SUCCESS: zfs set volmode=on testpool/vol exited 255
  02:13:24.27 SUCCESS: zfs set volmode=1 testpool/vol exited 255
  02:13:24.27 SUCCESS: zfs set volmode=nope testpool/vol exited 255
  02:13:24.28 SUCCESS: zfs set volmode=- testpool/vol exited 255
  02:13:24.30 SUCCESS: zfs set volmode=none testpool/vol
  02:13:24.32 SUCCESS: zfs destroy testpool/vol
  02:13:24.35 SUCCESS: zfs create -V 256m -s testpool/vol
  02:13:24.38 SUCCESS: zfs set volmode=full testpool/vol
  02:13:24.38 SUCCESS: dd if=/dev/zvol/testpool/vol of=/dev/null count=1
  02:13:24.42 SUCCESS: dd if=/dev/zero of=/dev/zvol/testpool/vol count=1
  02:13:24.44 zvol/testpool/vol created
  02:13:24.47 zvol/testpool/volp1 added
  02:13:24.47 SUCCESS: verify_partition /dev/zvol/testpool/vol
  02:13:24.57 SUCCESS: zfs set volmode=geom testpool/vol
  02:13:24.[60](https://github.com/openzfs/zfs/actions/runs/11420934502/job/31777190630?pr=16625#step:11:61) SUCCESS: zfs destroy testpool/vol
  02:13:24.63 SUCCESS: zfs create -V 256m -s testpool/vol
  02:13:24.65 SUCCESS: zfs set volmode=dev testpool/vol
  02:13:40.30 /dev/zvol/testpool/vol does not exist as a block device

[amotin]

I believe the problem is valid and caused by the race between old devfs entry removal on asynchronous GEOM_DEV orphan call by volmode=geom code and synchronous creation of new entry by volmode=dev code. I was thinking we could just call g_waitidle() to make sure all async events have settled, but seems like it waits only for g_events events, but not for g_doorstep ones used by orphan processing. So it might require patching FreeBSD first, or looking for other mechanism, like sending event to ourselves and waiting for it.

[naY9yjoS6ZqhOd35sIFH]

it seems reproduced in normal using with zfs-2.3.0 and 2.3.1 linux 6.12 as zfs rollback zvol snapshot which volsize smaller than snapshot which is large volsize,and hang-up at pre volsize which seems not change back,and have to reboot to clear hang-up state

[amotin]

@naY9yjoS6ZqhOd35sIFH I don't see how it is related here. If what you describe is true, it is a different issue.