F40: `failed to write bootloader configuation` when installing from Anaconda ISO

[miabbott]

The OpenQA tests have discovered that the Anaconda ISO install path is failing with the error failed to write bootloader configuation

https://openqa.fedoraproject.org/tests/2857297#step/_do_install_and_reboot/115

It's believed that the most recent greenboot update (https://bodhi.fedoraproject.org/updates/FEDORA-2024-1d32971a85) is the likely culprit.

It is pulling in bootupd as part of the Requires (see fedora-iot/greenboot#129) and the Anaconda installer is trying to use bootupd to install the bootloader (https://github.com/rhinstaller/anaconda/blob/443d5e6d7e3f97477dbdc870d73957fcc34d0fc5/pyanaconda/modules/payloads/payload/rpm_ostree/installation.py#L504-L509).

Fedora IoT 40 isn't equipped to support bootupd at this time, so we likely need to remove the Requires: bootupd from the spec file

[cgwalters]

One thing I'll point out here is that I think IoT should stop being its own base image and build on top instead of fedora-bootc - the bootupd integration is already tested and working there and it's not a really good idea to try to re-do that I would say.

I think we can refactor things such that the bootupd integration is only enabled in greenboot when installed on a derivation of fedora-bootc.

For example, the container image could do something like this:

FROM quay.io/fedora/fedora-bootc:41
RUN dnf -y install greenboot && mv /etc/grub2.d/greenboot.conf /usr/lib/bootupd/grub2-static/configs.d

or so. Or, the more painful way would be to try to split the package. Or, change it so it doesn't Requires: bootupd but still unconditionally installs the static config (leaving the /etc/grub2.d as a symlink) - I think that'd be nearly equally easy.

[cgwalters]

As for why things are failing, it's probably because the IoT builds aren't doing https://gitlab.com/fedora/bootc/base-images/-/blob/main/tier-0/bootupd.yaml?ref_type=heads#L31 but again I think the better fix here is to get to a world of container derivation, not where IoT is its own base image.

[miabbott]

One thing I'll point out here is that I think IoT should stop being its own base image and build on top instead of fedora-bootc - the bootupd integration is already tested and working there and it's not a really good idea to try to re-do that I would say.

Let's explore that idea in a separate issue? Maybe #53 or #57? Or create a new one if you'd like

As for why things are failing, it's probably because the IoT builds aren't doing https://gitlab.com/fedora/bootc/base-images/-/blob/main/tier-0/bootupd.yaml?ref_type=heads#L31 but again I think the better fix here is to get to a world of container derivation, not where IoT is its own base image.

We've done that for F41 and F40, so I'm not sure why we are hitting the failure. Since we are breaking F40 builds, dropping bootupd from the greenboot spec feels like the quickest and easiest option right now (fedora-iot/greenboot#152).

[miabbott]

We've done that for F41 and F40, so I'm not sure why we are hitting the failure. Since we are breaking F40 builds, dropping bootupd from the greenboot spec feels like the quickest and easiest option right now (fedora-iot/greenboot#152).

Oh right, we also reverted it for F40 - https://pagure.io/fedora-iot/ostree/c/dc36d6caf078e60eebbf22b1786e53535aee7a15?branch=f40

So that explains why F40 was failing

[nullr0ute]

One thing I'll point out here is that I think IoT should stop being its own base image and build on top instead of fedora-bootc - the bootupd integration is already tested and working there and it's not a really good idea to try to re-do that I would say.

One issue is that fedora-bootc I believe is too fat for IoT.

[jlebon]

I opened https://gitlab.com/fedora/bootc/base-images/-/merge_requests/48 related to this. @nullr0ute @miabbott interested in your thoughts from the IoT side.

[miabbott]

@pcdubs please have a look at Jonathan's proposal above ⬆️

[travier]

Short term, I would remove the bootupd dependency in greenboot for Fedora 40.

Hopefully we keep bootupd enabled in F41 and it can stay there.

[travier]

Ah, that's what you did in https://src.fedoraproject.org/rpms/greenboot/c/e897d5d8709a64e405b22c9f347e1891b9f0306c?branch=f40 so LGTM.

[pcdubs]

As noted above, this was fixed and no longer an issue.