Upgrade path and state migration after v44 (datasets + new stage layout)

[kovagoadam]

I’d like to ask for clarification on how upgrades between Fabric releases are expected to be handled after the structural changes introduced post-v44 — specifically the introduction of the datasets mechanism and the reorganization of stages (e.g., 0-org-setup, new networking stage, etc.).

In earlier versions (before v44), there were explicit moved {} blocks within modules and stages to migrate Terraform state between versions — ensuring compatibility and smooth upgrades without the need to destroy and recreate existing resources.

However, starting from v44, these moved blocks seem to have been removed, and the new structure (datasets, reworked stages, etc.) does not appear to provide a direct migration path for existing Fabric deployments.

Observed behavior

• When trying to upgrade an existing Fabric environment (for example, from v43 to the current release):
• Resources in older stages like 0-bootstrap / 1-resman no longer map cleanly to the new stage layout (e.g., 0-org-setup).
• Terraform plans show complete replacement or deletion of previously managed resources, indicating that no state migration is taking place.
• Upgrading seems to require tearing down and re-applying all stages, which is not ideal for production environments.

Background and related context

I’ve found a few similar discussions in this repository, but most of them are over a year old, referring to the older Fabric architecture and pre-dataset module model.
I’m particularly interested in this question now because of:

• The new “factory-based” configuration model (e.g., project and network factories),
• The stage refactor (0-bootstrap → 0-org-setup, 1-resman merging logic, new networking layout, etc.),
• The new context based interpolations.

Given these major architectural shifts, it’s unclear whether Fabric expects in-place upgrades (with state migration between versions) or whether the expectation has changed toward fresh deployments per major revision.

Questions

• Is this expected behavior (i.e., upgrades across these structural boundaries are not supported)?
• If not, how does Fabric now handle state migrations between releases (since the moved blocks are gone)?
• What is the recommended approach to upgrade an existing Fabric deployment to the latest version — particularly for:
  1. 0-bootstrap → 0-org-setup
  2. The new networking stage
  3. Project-factory and similar modules

Any guidance or examples of a supported upgrade process (if one exists) would be greatly appreciated.

Example context

Previously deployed Fabric: v43
Attempted upgrade: v45+
Stages affected: 0-bootstrap-legacy, 1-resman-legacy, 2-networking, 3-project-factory

Expected outcome

Ideally, an existing Fabric deployment should be upgradeable (with possible manual steps) without full teardown/recreation — similar to the pre-v44 upgrade experience using moved {} blocks.

[ludoo]

We never explicitly supported upgrades, we've stated multiple times that:

• this repo is a toolkit developed for our own use which we share out of goodwill, but can't maintain as a product
• once a landing zone is complete there's no point upgrading anything other than modules, as by definition it meets original requirements
• once the development phase is finished, a landing zone becomes a living organism that does need to be maintained and evolved internally to reflect its users' needs, not a product lifecycle

With that in mind, we have started providing moved blocks in the past where that was possible because we were doing it anyway for our development orgs, but with no guarantees they would work, or be exhaustive. Given the scope of the changes in the last release, we are unable to do that and the only sane migration path is starting from scratch.

PS - Modules are a different thing of course, and there we add upgrading notes and moved blocks whenever possible, and plan on continuing doing so. But even there, some migrations are simply not possible to automate and that will never prevent us from improving interfaces or doing internal changes when those unlock new use cases.

[kovagoadam]

Thanks a lot for the detailed clarification — that makes perfect sense.
It’s good to understand that upgrades between Fabric releases aren’t meant to be a supported or guaranteed path, and that the previous moved {} blocks were more of a convenience for internal migrations rather than an official feature.

I really appreciate the insight about where upgrades are expected (within modules) and where the landing zone should instead evolve independently.
That clears up the intent behind the recent architectural changes — thank you for taking the time to explain it so clearly.

[ludoo]

No worries, it's a question we keep getting asked time and again. :) And while we would definitely like to allow easier upgrades, that is only possible when the codebase undergoes small incremental updates. Large, all-encompassing changes like the ones we did to FAST recently are basically impossible to migrate cleanly, especially in a generalized fashion.