Support for resuming from swap file on the LUKS encrypted root

[DmytroSytnyk]

Problem

Current version of ugrd ( ugrd-2.0.2 ) is not able to handle resuming from hibernate image located in swap file on the LUKS encrypted root partition (tested with swap file residing in the separate btrfs subvolume).
Normal image generation process works as expected in the case of normal booting (not resuming) but fails if the system is booted from the hibernated state.

The issue occurs on Gentoo with recent kernel (tested with gentoo-sources-6.16.4 and secure boot enabled).
Content of /etc/ugrd/config.toml:

modules = [
"ugrd.fs.resume",
]
kmod_autodetect_lspci = true

Image is generated using the sys-kernel/installkernel hook by invoking the usual make install from /us/src/linux.
Encrypted root and swap file within are created in accordance with BTRFS recommendations using the guide from Encrypted Root with LUKS and Opal.

Proper boot resume and resume_offset command-line options are supplied through the kernel config variable CONFIG_CMDLINE.

After booting from hibernated state the process get stuck inside handle_resume() portion of the image script informing that Resume device is not found.

Manual solution

Override the content of ugrd/fs/resume.toml by the following:

[imports.init_mount]
"ugrd.fs.resume" = [ "handle_resume" ]

[import_order.after]
handle_resume = "set_root_subvol"

[import_order.before]
handle_resume = "mount_root"

needed to make sure that the handle_resume() is called after the root partition was unlocked.

After regeneration of initramfs image the system is able to properly resume from the hibernated state.

More permanent fix would require implementation of the logic necessary to detect that the swap is located in file on the encrypted root.

[desultory]

The workaround you described is essentially the core of what is happening here: #290

this PR is a bit stalled because it needs more review and stuff for automated testing.

I'm hesitant to rush any of this because improper mounting or writing can potentially cause data loss when resuming. I think it should be safer on BTRFS than something like ZFS, but I'd like to take the time to give this stuff a thorough review before merging it

[DmytroSytnyk]

Of course. More testing is welcomed. I think blind changing of the file might brake the resuming functionality in other use cases.

[desultory]

Of course. More testing is welcomed. I think blind changing of the file might brake the resuming functionality in other use cases.

yeah I want to get it real ironed out, so maybe it can automatically enable that module if it sees resume stuff in the command line, or if it sees activated swap and the resume options available in /sys. I've been considering rolling it out as is with a manual toggle and warning about it being experimental. I really don't want to overlook something and eat someone's data. The bit that really makes me hesitate is that I don't use this functionality on any of my systems, so I want to make sure testing is very robust before I push it to everyone.

[DmytroSytnyk]

According to Kernel Documentation the suspend-from-file use case can be detected by the presence of resume_offset kernel parameter. If that option is detected along with the encrypted root, one can check if the swap file path from fstab/mtab is a subpath of the encrypted partition.

To facilitate more extensive testing one would need the capability to specify the kernel parameters from within ugRD.
Using kernel's CONFIG_CMDLINE for that is too slow.

I was unable to find how to specify kernel parameters in the existing ugRD examples.
Is it possible ?
efibootmgr seems to be supporting this functionality, although it is not clear how to see which parameters were specified for the given boot entry...

Regarding data corruption, the same doc says that kernel imposes some signature checks to avoid that.
So, as long as the suspend process is triggered via echo <Device_ID> > /sys/power/resume before anything is remounted, the data should be fine. I tested putting handle_resume() after mounting; then kernel simply refuses to suspend.

UPDATE 1:
It turns out that efibootmgr actually reports the supplied kernel parameters, but without -u option they are shown as as sequence of raw UTF-8 hexadecimals.
Piping through xxd -r -p | iconv -f UTF-8 reveals the text.

UPDATE 2:
After looking at EFI Stub generation script /usr/lib/kernel/install.d/95-efistub-kernel-bootcfg.install I realized how one can specify additional kernel boot parameters (either via /etc/default/uefi-mkconfig or /etc/kernel/cmdline depending on the installkernel configuration ).
To get verbose output of the installkernel one can do

INSTALLKERNEL_VERBOSE="1"  make install 

from /usr/src/linux.

I have successfully tested the same setup as explained before with XFS.

[bell07]

I was running into this issue. The grd tried to find the crypted-swap before luks handling.
From my point of view the resume handling should be always after crypto, if crypto is configured.

It is not recommended to leave swap not encrypted if root is encrypted for security reasons.
If anyone uses this, then the not encrypted swap is used after root decryption, so it still working.
Maybe it make sense to show a warning to user with recommendation to encrypt the swap for proper data security.

[desultory]

I was running into this issue. The grd tried to find the crypted-swap before luks handling. From my point of view the resume handling should be always after crypto, if crypto is configured.

the issue with this is cases where there is nothing to resume from. If your swap is on another volume entirely, you'll be entering keys for no reason pretty often. With encrypted swap especially, it's difficult to know if it should actually be used, unless the user is doing something like using a different boot entry with a resume= parameter specifically when they want to resume.

It is not recommended to leave swap not encrypted if root is encrypted for security reasons. If anyone uses this, then the not encrypted swap is used after root decryption, so it still working. Maybe it make sense to show a warning to user with recommendation to encrypt the swap for proper data security.

I agree, but right now ugrd doesn't handle encrypted swap fully automatically, so this would be pushing users to a half supported method. Once there is full test coverage then this makes sense to suggest.

[DmytroSytnyk]

I think the case with encrypted swap on a separate partition is intrinsically not safe. The resume signature has a fixed position in swap and therefore one can corrupt the swap while preserving the signature, or fake the hibernated state. This is harder with swap on encrypted root (even though the position of the signature is also known) because the access to this file can be more granulary controlled.
One can even mark it immutable/read-only for the user-space and the hibernation would still work.

[julie-de-ville]

Manual solution

Override the content of ugrd/fs/resume.toml by the following:

[imports.init_mount]
"ugrd.fs.resume" = [ "handle_resume" ]

[import_order.after]
handle_resume = "set_root_subvol"

[import_order.before]
handle_resume = "mount_root"

Did you have to do anything else to get resume working? I have a similar setup: btrfs root and swap partitions on lvm in luks container. I added resume module, the config you posted, and set resume=UUID=<swap-uuid> in /etc/default/uefi-mkconfig cmdline, but I cannot get it to show resume in /sys/power, nor can I add 'disk' to /sys/power/state.

[DmytroSytnyk]

No, I did not change anything else within ugrd. It is not clear for me, however, what do you mean by <swap-uuid>. In order for resume to work you need to provide two kernel parameters resume with the partition and resume_offset for the offset to tmp file residing on this partition (see the very first post here and the accompanied guide).
I pivoted away from BTRFS because the I/O performance was very low, but now use the same setup with tmp file on XFS root. The I/O performance is roughly 1.8 times higher (write and rewrite is ~2 times higher and read is 1.1-1.2 lower).
How do you generate the initramfs image ?

[julie-de-ville]

Oh I wasn't sure if I needed to set resume_offset because it is a swap partition on an lvm. I have resume=UUID=<uuid> of (my swap partition) in my cmdline at /etc/default/uefi-mkconfig

I generate my initramfs by running installkernel after enabling ugrd as a use flag for the package atom

[DmytroSytnyk]

Swap partition should work with the original ugrd config.
The situation discussed in the ticket deals with the swap in a file on the encrypted root.

Perhaps with BTRFS there is also a possibility to have swap on the sub-volume. I never tried this. Do you use opal hardware disk encryption or the software one ?

[julie-de-ville]

I am using btrfs+swap in lvm in a luks container. I have resume set in the cmdline, but if I do not enable the module explicitly in ugrd, it does not seem to parse it.

I also get this error
No initramfs fstab found, skipping mount_fstab. If non-root storage devices are not needed at boot, this is fine.