Intended bootupd integration?

[Mstrodl]

Hey all,

Sorry if this is already well-documented somewhere (or if this really belongs in the bootupd repo, I wasn't sure...). Right now, at the company I work for, we use ostree to manage images for a bunch of debian-based appliances. When we want to install an update, we run ostree admin upgrade -r. Pretty simple. I did look at moving to bootc as @cgwalters has pointed out it's The Future™ last time I talked about our system, but making that work on debian seemed kinda daunting and not worth the payoff at the time.

Now that Debian Trixie is released though, it seems like we're running into issues with our old EFI images...
I need to dig some more, but I think what's happening is that the new grub configs generated by trixie are incompatible with the old EFI images from bookworm. As a result, the first upgrade works fine, but the next upgrade after that (when configs get regenerated) will brick the bootloader.

Maybe I'm missing something, but it doesn't seem like this will call out to bootupd to update EFI images first... right? How is this supposed to work? I imagine you want to have ostree pull in the new content, do a checkout or otherwise copy the new bootloader images (for bootupd), have bootupd do its update, then carry on with the deploy as usual.

That question aside: I understand there are big things happening on bootc to decouple it from rpm, but it seemed like they were blocked on fedora 43 changing the bootloader file structure. I was going to just modify bootc to either get its version information from the directory structure debian has in place, get version data from dpkg, or just always apply bootloader updates (so long as they're atomic using EFI variables it seems pretty safe, no?). These would probably end up being in a fork since I don't want to screw up the changes you guys have planned for fedora, but if there's a better way to do it today (or if what I'm doing still makes sense to upstream), by all means let me know :)

Thanks!

[champtar]

For the bootupd part, using version 0.2.28, you can have a systemd unit

[Unit]
Description=bootloader adopt
ConditionKernelCommandLine=ostree
ConditionPathExists=!/boot/bootupd-state.json
RequiresMountsFor=/sysroot /boot
Before=bootloader-update.service

[Service]
Type=oneshot
ExecStart=/usr/bin/bootupctl adopt-and-update --with-static-config
RemainAfterExit=yes
# Keep this stuff in sync with SYSTEMD_ARGS_BOOTUPD in general
PrivateNetwork=yes
ProtectHome=yes
KillMode=mixed
MountFlags=slave

[Install]
WantedBy=multi-user.target

It'll update grub, the grub config, and set ostree to not generate grub config anymore (just the BLS files)

[champtar]

VFAT supports atomic swapping of directories, but you would need to place everything in 1 directory (right now it's in 3, efi/grub2/loader).
Hopefully UKI will simplify all of this.

[Mstrodl]

Right... UKI will fix a lot of this, I think.

I've realized doing it at startup still has brick potential: downgrade to older version (grub config generated with booted deployment.. why?), it boots, old bootloader is applied, reboot, boot fails due to new grub options introduced in the userspace grub tools in original deployment. I'm sure the reverse is possible too (old grub config generated contains deprecated option removed in new grub efi images).

What I'm doing for now is basically to add two small hooks to the bootswap which trigger my bootloader swap code:

diff --git a/src/libostree/ostree-sysroot-deploy.c b/src/libostree/ostree-sysroot-deploy.c
index 18be5620..78fe7ae9 100644
--- a/src/libostree/ostree-sysroot-deploy.c
+++ b/src/libostree/ostree-sysroot-deploy.c
@@ -2415,6 +2415,8 @@ write_deployments_bootswap (OstreeSysroot *self, GPtrArray *new_deployments,
                             SyncStats *out_syncstats, char **out_subbootdir,
                             GCancellable *cancellable, GError **error)
 {
+  gboolean bootplugin_exists;
+  int estatus;
   const int new_bootversion = self->bootversion ? 0 : 1;

   g_autofree char *new_loader_entries_dir
@@ -2466,12 +2468,37 @@ write_deployments_bootswap (OstreeSysroot *self, GPtrArray *new_deployments,
   if (!prepare_new_bootloader_link (self, self->bootversion, new_bootversion, cancellable, error))
     return FALSE;

+  const char *const bootplugin_deploy_argv[] = { "/usr/lib/ostree/bootloader-plugin", "deploy", NULL };
+  bootplugin_exists = g_file_test(bootplugin_deploy_argv[0], G_FILE_TEST_EXISTS);
+  if (bootplugin_exists)
+    {
+      if (!g_spawn_sync(NULL, (char **) bootplugin_deploy_argv, NULL, G_SPAWN_CHILD_INHERITS_STDERR,
+                        NULL, NULL, NULL, NULL, &estatus, error)) {
+        return glnx_prefix_error (error, "bootloader-plugin deploy exec failed");
+      }
+      if (!g_spawn_check_exit_status (estatus, error)) {
+        return glnx_prefix_error (error, "bootloader-plugin deploy failed");
+      }
+    }
+
   if (!full_system_sync (self, out_syncstats, cancellable, error))
     return FALSE;

   if (!swap_bootloader (self, bootloader, self->bootversion, new_bootversion, cancellable, error))
     return FALSE;

+  const char *const bootplugin_swap_argv[] = { bootplugin_deploy_argv[0], "swap", NULL };
+  if (bootplugin_exists)
+    {
+      if (!g_spawn_sync(NULL, (char **) bootplugin_swap_argv, NULL, G_SPAWN_CHILD_INHERITS_STDERR,
+                        NULL, NULL, NULL, NULL, &estatus, error)) {
+        return glnx_prefix_error (error, "bootloader-plugin swap exec failed");
+      }
+      if (!g_spawn_check_exit_status (estatus, error)) {
+        return glnx_prefix_error (error, "bootloader-plugin swap failed");
+      }
+    }
+
   if (out_subbootdir)
     *out_subbootdir = g_steal_pointer (&new_subbootdir);
   return TRUE;
--
2.39.5 (Apple Git-154)

[champtar]

grub config generated with booted deployment.. why?

If you don't use secure boot you might consider rebuilding grub with BLS support and using static grub config, it'll allow to run less code during upgrade instead of more, and you can update grub and grub config at the same time, you don't need to change grub config with each deployment, just the BLS files.

[champtar]

systemd-boot (supports BLS) is not yet supported by ostree but there is a PR for it

[Mstrodl]

Yeah when I set this up originally I was wanting to do systemd-boot. I think grub is a bigger hammer than we really need :)