Initial factory reset code #1588

ckyrouac · 2025-09-05T15:23:46Z

This is a rebased version of #1389 with some fixes on top to make bootc install reset --experimental work with the latest soft reboot code. I'd like to get this in hidden behind --experimental, then iterate on it. I plan to add some e2e tests as a followup but I can work on including them as part of this PR if you want.

gemini-code-assist

Code Review

This pull request introduces a new experimental bootc install reset command, which appears to function as a factory reset. The implementation is well-structured, refactoring existing deployment logic to use a new MergeState enum, which is a clean way to handle different deployment sources. The changes also include new utilities for managing stateroots. I've found a couple of issues that need to be addressed before merging.

cgwalters · 2025-09-05T18:03:04Z

I think we'll want tests at least before merging, and supporting that with tmt is probably going to be an excellent driver for the feature.

I haven't dug in but I think tmt is relying on injecting ssh keys w/cloud-init, so it might Just Work to do a reset there?

ckyrouac · 2025-09-24T15:03:18Z

I think we'll want tests at least before merging

Updated with a basic test.

cgwalters · 2025-09-26T14:38:26Z

crates/lib/src/deploy.rs

+        let deployment_dir = ostree.deployment_dirpath(&staged);
+        let deployment_dir = std::path::Path::new(deployment_dir.as_str());
+        if deployment_dir.exists() {
+            let marker_path = deployment_dir.join(".bootc-factory-reset");


Let's have clear const for things like this with an explanation there.

But backing up a second, in the ostree side we have "deployment backing" directories already where we can put metadata.

Though of course we should also be thinking about how this would work with the composefs backend.

Hmmm hmm. Here's an idea, how about we use the deployment's etc directory for state like this instead? I think we could save e.g. system.bootc.merged_from xattr or so?

Basically I lean towards extended attributes over "stamp files" for metadata.

BTW though do note the intersection here with etc.transient - I wonder if we should make that more visible in status output too. If that's enabled each upgrade is already like a reset of /etc.

crates/lib/src/status.rs

cgwalters · 2025-09-26T14:41:09Z

crates/lib/src/status.rs

+    if is_factory_reset {
+        // Factory reset deployments don't support soft reboots
+        // this is primarily because the kargs validation will fail when checking for soft reboot
+        // compatibility in the ostree code, which will cause bootc status and upgrade to fail.


I don't understand why we need to special case this. Why is bootc status failing exactly?

If we're changing (usually dropping) kernel arguments won't the existing karg check work?

As a followup - we talked about this in a live chat and there seems to be a libostree bug here where the staged deployment is missing ostree= and that trips an assertion in the soft reboot checks.

Ideally we root cause and fix that.

But we may be able to work around this by detecting when there's no kargs in the staged deployment?

I think this will fix the factory resest issue: ostreedev/ostree#3532. Still investigating why the deployment is missing the ostree arg.

jlebon · 2025-10-07T14:00:24Z

Just saw the demo for this. A few random thoughts.

It'd be nice to be able to have different reset levels. One is just resetting /etc. This doesn't require a new stateroot. Another would be resetting both /etc and /var.

Should this use stateroots at all? This would be another thing to implement in the composefs backend (or maybe that code and concept already exist there?). But also it's quite common for /var or subdirectories of /var to be mounts. I.e. the stateroot var is not used in that case. We should probably detect this and error out.

One alternative though is basically to just move everything in /var into /var/.previous. That would work in both cases, is cheap, and makes access to the previous /var trivial.

(There's still the case of submounts in /var of course. We could apply that rename strategy recursively, or just warn and leave it up to the user -- or require them to be unmounted before proceeding?)

Signed-off-by: Colin Walters <[email protected]>

This is a nondestructive variant of `to-existing-root`. Signed-off-by: Colin Walters <[email protected]>

Also cleans up some bugs from rebasing previous commits. Signed-off-by: ckyrouac <[email protected]>

Signed-off-by: ckyrouac <[email protected]>

cgwalters

Looks like there's some CI failures, but otherwise we could try to get this in?

It'd be a bit ugly but maybe we could detect the ostree version and avoid crashing if it's too old?

Dockerfile

Add ostree version check to has_soft_reboot_capability() to ensure soft reboot is only enabled when ostree 2025.7 or later is present. This prevents attempting soft reboots on older ostree versions that have a bug when validating kargs during a factory reset. The check happens before the systemd and deployment capability checks, providing a fast path return for incompatible versions. Signed-off-by: ckyrouac <[email protected]>

ckyrouac · 2025-10-20T15:23:07Z

It'd be a bit ugly but maybe we could detect the ostree version and avoid crashing if it's too old?

Just pushed up a commit to check the ostree version in the has_soft_reboot_capability function where the crash happens. I haven't had a chance to dig into the deeper root cause of why the ostree= karg is getting set but the version check should unblock this PR's CI. The result is on systems with an outdated ostree, soft reboot will be disabled for factory reset deployments. If it works for you, I'll just file an issue to track fixing the ostree= karg issue.

cgwalters

Looks overall sane to me!

One thing that would probably help is to add docs for this in our experimental section.

cgwalters · 2025-10-20T19:35:30Z

tmt/tests/booted/test-factory-reset.nu

+    assert not equal $reset_status.status.otherDeployments.0.ostree.stateroot "default"
+
+    # we need tmt in the new stateroot for second_boot
+    print "Copying tmt into new stateroot"


I guess this one is really asking for enhancing reset to support copying data in a nicer way.

working on adding bootc mount

ckyrouac requested a review from cgwalters September 5, 2025 15:23

ckyrouac force-pushed the factory-reset branch from 387f495 to ab75caf Compare September 5, 2025 15:25

gemini-code-assist bot reviewed Sep 5, 2025

View reviewed changes

ckyrouac marked this pull request as draft September 8, 2025 12:52

ckyrouac force-pushed the factory-reset branch 5 times, most recently from b45e272 to a0cd215 Compare September 24, 2025 13:37

ckyrouac marked this pull request as ready for review September 24, 2025 15:02

bootc-bot bot requested a review from jeckersb September 24, 2025 15:02

ckyrouac force-pushed the factory-reset branch from a0cd215 to 2ad17c6 Compare September 25, 2025 17:21

cgwalters requested changes Sep 26, 2025

View reviewed changes

ckyrouac force-pushed the factory-reset branch from 2ad17c6 to 6003aa7 Compare October 9, 2025 16:04

github-actions bot added area/install Issues related to `bootc install` area/ostree Issues related to ostree labels Oct 9, 2025

ckyrouac force-pushed the factory-reset branch 3 times, most recently from be8b439 to 46e0668 Compare October 15, 2025 16:06

cgwalters and others added 4 commits October 15, 2025 12:35

ostree-ext: Add sysroot listing + creation helpers

c3e9d72

Signed-off-by: Colin Walters <[email protected]>

install: Add reset

cf12d52

This is a nondestructive variant of `to-existing-root`. Signed-off-by: Colin Walters <[email protected]>

reset: Move kernel code into bootc_kargs

b9aa9c4

Also cleans up some bugs from rebasing previous commits. Signed-off-by: ckyrouac <[email protected]>

reset: Initial tmt test for factory reset

c3bf2ad

Signed-off-by: ckyrouac <[email protected]>

ckyrouac force-pushed the factory-reset branch from 46e0668 to def648d Compare October 15, 2025 16:35

cgwalters mentioned this pull request Oct 17, 2025

install: Add reset #1389

Closed

cgwalters reviewed Oct 17, 2025

View reviewed changes

Dockerfile Outdated Show resolved Hide resolved

Dockerfile Outdated Show resolved Hide resolved

ckyrouac force-pushed the factory-reset branch from def648d to ea5896f Compare October 20, 2025 15:15

cgwalters approved these changes Oct 20, 2025

View reviewed changes

Initial factory reset code #1588

Are you sure you want to change the base?

Initial factory reset code #1588

Uh oh!

Conversation

ckyrouac commented Sep 5, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

cgwalters commented Sep 5, 2025

Uh oh!

ckyrouac commented Sep 24, 2025

Uh oh!

cgwalters Sep 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cgwalters Sep 26, 2025

Choose a reason for hiding this comment

Uh oh!

cgwalters Oct 8, 2025

Choose a reason for hiding this comment

Uh oh!

ckyrouac Oct 9, 2025

Choose a reason for hiding this comment

Uh oh!

jlebon commented Oct 7, 2025

Uh oh!

cgwalters left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ckyrouac commented Oct 20, 2025

Uh oh!

cgwalters left a comment

Choose a reason for hiding this comment

Uh oh!

cgwalters Oct 20, 2025

Choose a reason for hiding this comment

Uh oh!

ckyrouac Oct 20, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

cgwalters Sep 26, 2025 •

edited

Loading