Versioned bundle and Execution receipt #3623

vedhavyas · 2025-07-09T10:11:49Z

First of all, changes may appear big but most of the changes are cosmetic.
I did not change everything at once but rather inroduced necessary intermediate types so that I'm aware which parts require direct changes, like runtime, and which parts require an actual compatilibility code. These intermediate types would have be removed in the following commits.

Notable changes

New Bundle V1 version
New ER version V0
New FraudProof V1 version that will hold both old and new Bundle Versions.

For the reviewers, you can either follow commit by commit to see how the migration is applied or look at overall changes. Overall changes are surprisingly easy to understand.

Please let me know @NingLin-P if there are changes that missed that may cause incomaptibility on current Taurus.

Cleanup

We can safely remove

Bundle V0
FraudProof V0
Once the new traurus is deployed. As for the mainnet, assuming, this PR lands before we instantiate domains this should be safe to remove as well.

Closes: #2942

Code contributor checklist:

I have read, understood and followed contributing guide

…y code for previous fraud proof version

teor2345

Looks good overall, but it's a large change. What is our testing and risk mitigation strategy?

There are a lot of new functions and types without documentation. Can you add a short description to the production ones? (Tests aren't as important, but it can still be useful to say what a test is meant to check.)

What does a version upgrade look like? Do we need documentation that says how to do it, and tests to make sure it works? (Or a documented manual test process.)

There are some duplicated code blocks and types we could fix up, to avoid confusion.

crates/pallet-domains/src/block_tree.rs

crates/sp-domains-fraud-proof/benches/fraud_proof_verification.rs

domains/client/domain-operator/src/aux_schema.rs

vedhavyas · 2025-07-10T02:09:24Z

What is our testing and risk mitigation strategy?

Same as usual.

Deploy Local testnet with latest client but older runtime and do an upgrade.
Sync taurus node from scratch with latest client.

Both should cover all the cases

What does a version upgrade look like? Do we need documentation that says how to do it, and tests to make sure it works? (Or a documented manual test process.)

no, its purely defined by the runtime and client uses whatever runtime expects. Only thing we should aware is the node release should go in first followed by runtime upgrade for Taurus. For mainnet, since there are no domains, we can release a client as soon as taurus is tested.

teor2345 · 2025-07-10T02:38:37Z

no, its purely defined by the runtime and client uses whatever runtime expects. Only thing we should aware is the node release should go in first followed by runtime upgrade for Taurus.

These things seem important to document somewhere, because we'll be doing another upgrade on mainnet with domains at some point in the future.

vedhavyas · 2025-07-10T06:20:09Z

These things seem important to document somewhere, because we'll be doing another upgrade on mainnet with domains at some point in the future.

Not necessarily. Take a look at how XDM is versioned. This is same as that. Runtime defines the versions that need to be used and client uses that.

As for protocol specs, there are no changes to the specs since nothing has changed but rather made type versioned

…emove unnecessary clone

crates/pallet-domains/src/lib.rs

NingLin-P

There are a lot of changes and a lot of duplicated code, I don't have the confidence to find every issue (if any) with my bare eye, thus let's see how the test goes.

crates/pallet-domains/src/lib.rs

vedhavyas · 2025-07-14T06:51:32Z

@NingLin-P I think there maybe confusion on Bundle version and Er version when it comes to storing them

A runtime will only accept the Bundle version that is defined on the Runtime's CurrentVersion.
If a runtime says it expects V1 bundle version, then bundle submission must be of version V1 and not any other.

Now coming to ER version, it changes a bit.
ER version is defined based on the consensus block number at which ER is derived
So if the Runtime's ER version at the time of deriving block is V0, then runtime will accept V0 ER when that ER is submitted through the bundle or singleton receipt since between ER is derived and Submitted there maybe a Runtime upgrade and ER version would have changed.

now coming to how we store the versions, we hook into the set_code
When we initiate the runtime upgrade, example spec_version 2, set_code is called on runtime with spec_version 1
Spec version 1's current versions are stored against the block number which included set_code extrinsics. So any ERs derived until the upgraded number should come in the same ER version as spec version 1 ER version
From upgraded_number + 1, ER version may have changed, and all the ER derived from that number should come with version accepted by new Runtime.

I have added tests to explain this better.

Let me know if this is clear or we can do a sync call

Note: conflicts will be fixed once we have an approval from the team else merge commits will pollute the actual commits from this PR

…ck at which rutime upgrade did takes place

vedhavyas

More specifically, when the ER version is upgraded from V0 to V1 in consensus block #N, the runtime will assume using V0 for block #N and using V1 starting from block #N+1. While on the client side, when querying the ER versions in block #N it will return V1, thus it will use V1 to construct ER derived from #N. This inconsistency will cause the bundle to be rejected and the domain chain to stop progressing.

Thanks for pointing it out @NingLin-P . This was indeed missed from my end. Should be fixed in the new commits.

I wonder if it is better to do ongoing maintenance of the current code. I think there are risks in merging within the next week (or two). And risks and costs in delaying other plans to fit it in.

@teor2345 The whole reason why the compatibility code is removed to actually push to devnet and subsequently to mainnet since we dont have a tarus beyond this point.

Another way to reduce the risk is splitting the PR into:

I do not agree on this. IF a PR changes or introduces somethings, then its tests and everything affected by it should be part of the PR. We did this earlier and it did not bode well especially its multiple steps of opening a new PR and context switching to something else.

I would rather have reviewers take as much as they need to understand the changes and the side-effects it brings, if any, before merging the PR or else its a No-go from my end.

crates/pallet-domains/src/benchmarking.rs

crates/pallet-domains/src/block_tree.rs

crates/sp-domains-fraud-proof/src/lib.rs

teor2345 · 2025-07-18T00:20:01Z

Another way to reduce the risk is splitting the PR into:

I do not agree on this. IF a PR changes or introduces somethings, then its tests and everything affected by it should be part of the PR. We did this earlier and it did not bode well especially its multiple steps of opening a new PR and context switching to something else.

Hmm, I'm not sure if there's been a misunderstanding here.

Why are tests needed for "refactors that do not change functionality at all"?

teor2345

I have some non-blocking code style questions.

There will be some residual risk no matter how much we review this. Let's do some initial testing, and see if any bugs remain?

That will give us a better idea of the risk of this change, and we can decide what to do next when we know more.

crates/pallet-domains/src/lib.rs

teor2345 · 2025-07-18T00:48:58Z

crates/pallet-domains/src/lib.rs

+    pub(crate) fn set_previous_bundle_and_execution_receipt_version<SV, PV, BEV>(
+        block_number: BlockNumberFor<T>,
+        set_version: SV,
+        previous_versions: PV,
+        current_version: BEV,
+    ) where
+        SV: Fn(BTreeMap<BlockNumberFor<T>, BEV>),
+        PV: Fn() -> BTreeMap<BlockNumberFor<T>, BEV>,
+        BEV: PartialEq,
+    {
+        let mut versions = previous_versions();


This is an unusual code style.

Normally we would just pass a BTreeMap<BlockNumberFor<T>, BEV> directly to the function, return a BTreeMap<BlockNumberFor<T>, BEV> from the function, and then use the return value to set the storage.

Since the get and set are called unconditionally, I'm not sure why we're passing Fns to this function.

Passing Fns will reduce the amount of inlining and optimisation the compiler can do, potentially reducing performance (or increasing code size).

explained why I did above

teor2345 · 2025-07-18T00:49:52Z

crates/pallet-domains/src/lib.rs

+    pub(crate) fn bundle_and_execution_receipt_version_for_consensus_number<PV, BEV>(
+        er_derived_number: BlockNumberFor<T>,
+        previous_versions: PV,
+        current_version: BEV,
+    ) -> Option<BEV>
+    where
+        PV: Fn() -> BTreeMap<BlockNumberFor<T>, BEV>,
+        BEV: Copy + Clone,
+    {
+        let versions = previous_versions();


Similar feedback here, normally we would just pass BTreeMap<BlockNumberFor<T>, BEV> to this function directly.

Passing a Fn will reduce the amount of inlining and optimisation the compiler can do, potentially reducing performance (or increasing code size).

This is done specifically for testing. PTAL at testing this logic with mock versions and mock storage but reusing the same the same logic to pick the correct versions

I made these comments based on the test code.

What stops us calling the production code as:

bundle_and_execution_receipt_version_for_consensus_number( er_derived_number, CurrentBundleAndExecutionReceiptVersion::get(), PreviousBundleAndExecutionReceiptVersions::get(), )

And the test code as:

bundle_and_execution_receipt_version_for_consensus_number( er_derived_number, MockCurrentBundleAndExecutionReceiptVersion::get(), MockPreviousBundleAndExecutionReceiptVersions::get(), )

This is due to the change in type between production and testing for BundleAndExecutionReceiptVersion and in the inner types.

I think we might be talking about different things here, I'll open a PR to show what I mean.

See PR #3646, particularly the first commit 9f50a8a

teor2345

It appears 3 benchmarks are broken now:

- pallet_messenger_from_domains_extension::from_domains_relay_message
- pallet_messenger_from_domains_extension::from_domains_relay_message_channel_open
- pallet_messenger_from_domains_extension::from_domains_relay_message_response

https://github.com/autonomys/subspace/actions/runs/16361523792/job/46230168298?pr=3623#step:6:5460

vedhavyas · 2025-07-18T03:52:46Z

Nice! very helpful. Thanks 🙏🏼

teor2345 · 2025-07-22T21:41:19Z

crates/subspace-node/src/domain/auto_id_chain_spec.rs


+/// Returns AutoId genesis domain.
+/// Note: Currently unused since dev or devnet uses EVM domain and not AutoId
+#[allow(dead_code)]


If this code is always unused, you could mark it as:

Suggested change

#[allow(dead_code)]

#[expect(dead_code)]

Then if we use it in future, we'll get a compile error and remove that annotation.

Moved to PR #3646

teor2345

Looks good, thanks!

vedhavyas added 9 commits July 3, 2025 14:19

introduce v1 bundle version and rename current bundle type to BundleV1

dbf7599

introduce fraud proof v1 and update fraud proof api with compatibilit…

c1fc6d6

…y code for previous fraud proof version

move fraud proof v0 to its own module

15b37a3

refactor bundle versions and fraud proof versions to specific modules

7aa3c95

move execution receipt type to its own module

3361109

define current execution receipt version and update getters

8af40f8

rename execution receipt to v0

05b9593

cleanup intermediate code and add V0 bundle version to the bundle type

7367a2c

add v0 of execution receipt and versioned Er for Bundle V1

8469518

vedhavyas requested review from NingLin-P and nazar-pc as code owners July 9, 2025 10:11

vedhavyas requested a review from teor2345 July 9, 2025 10:12

vedhavyas force-pushed the versioned_bundle branch from 5e66e75 to 335e1d6 Compare July 9, 2025 10:13

teor2345 reviewed Jul 9, 2025

View reviewed changes

crates/pallet-domains/src/block_tree.rs Show resolved Hide resolved

crates/sp-domains-fraud-proof/benches/fraud_proof_verification.rs Show resolved Hide resolved

domains/client/domain-operator/src/aux_schema.rs Outdated Show resolved Hide resolved

add runtime compatibility code in decoding ER and BlockTreeNode and r…

77abccd

…emove unnecessary clone

vedhavyas force-pushed the versioned_bundle branch from 335e1d6 to 77abccd Compare July 10, 2025 06:38

update docs

4980b06

teor2345 added the needs benchmarking Significant benchmark or runtime code changes, re-run benchmarks before the runtime release label Jul 10, 2025

NingLin-P reviewed Jul 11, 2025

View reviewed changes

crates/pallet-domains/src/lib.rs Show resolved Hide resolved

crates/pallet-domains/src/lib.rs Show resolved Hide resolved

vedhavyas added 3 commits July 11, 2025 15:57

store previous bundle and er versions on runtime upgrade

87885e4

use execution receipt version on client when deriving new ER

93f0d08

check expected execution receipt version

3fa7be6

vedhavyas requested a review from NingLin-P July 11, 2025 11:27

NingLin-P reviewed Jul 11, 2025

View reviewed changes

crates/pallet-domains/src/lib.rs Outdated Show resolved Hide resolved

crates/pallet-domains/src/lib.rs Show resolved Hide resolved

crates/pallet-domains/src/lib.rs Outdated Show resolved Hide resolved

add tests for storing versions and quering versions

fea69d7

vedhavyas force-pushed the versioned_bundle branch from 86f298b to fea69d7 Compare July 14, 2025 06:53

ensure correct execution and bundle versions are returned for the blo…

0391dcb

…ck at which rutime upgrade did takes place

vedhavyas force-pushed the versioned_bundle branch from 96df613 to 40c14ef Compare July 17, 2025 04:42

update comments, optimisations, and code reverts

5ed52a2

vedhavyas force-pushed the versioned_bundle branch from 40c14ef to 5ed52a2 Compare July 17, 2025 04:54

vedhavyas commented Jul 17, 2025

View reviewed changes

crates/pallet-domains/src/benchmarking.rs Show resolved Hide resolved

crates/pallet-domains/src/block_tree.rs Show resolved Hide resolved

crates/sp-domains-fraud-proof/src/lib.rs Show resolved Hide resolved

Merge branch 'main' into versioned_bundle

75bf1b4

vedhavyas requested review from NingLin-P, jfrank-summit and teor2345 July 17, 2025 05:09

teor2345 previously approved these changes Jul 18, 2025

View reviewed changes

refactor variable names

4bafe4b

vedhavyas dismissed teor2345’s stale review via 4bafe4b July 18, 2025 03:33

vedhavyas enabled auto-merge July 18, 2025 03:33

vedhavyas requested a review from teor2345 July 18, 2025 03:34

teor2345 reviewed Jul 18, 2025

View reviewed changes

vedhavyas added 3 commits July 18, 2025 09:22

Merge branch 'main' into versioned_bundle

2840651

update benchmark fixtures for messenger

a38c00e

update devnet chain spec

90aec04

jfrank-summit added this to Security Audit (PRs) Jul 22, 2025

github-project-automation bot moved this to Backlog in Security Audit (PRs) Jul 22, 2025

teor2345 reviewed Jul 22, 2025

View reviewed changes

teor2345 approved these changes Jul 22, 2025

View reviewed changes

vedhavyas added this pull request to the merge queue Jul 22, 2025

Merged via the queue into main with commit ab4fdcc Jul 23, 2025
19 checks passed

vedhavyas deleted the versioned_bundle branch July 23, 2025 00:53

mmostafas moved this from Backlog to In progress in Security Audit (PRs) Jul 28, 2025

mmostafas moved this from In progress to Audited in Security Audit (PRs) Aug 14, 2025

Versioned bundle and Execution receipt #3623

Versioned bundle and Execution receipt #3623

Uh oh!

Conversation

vedhavyas commented Jul 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Notable changes

Cleanup

Code contributor checklist:

Uh oh!

teor2345 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vedhavyas commented Jul 10, 2025

Uh oh!

teor2345 commented Jul 10, 2025

Uh oh!

vedhavyas commented Jul 10, 2025

Uh oh!

Uh oh!

Uh oh!

NingLin-P left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vedhavyas commented Jul 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vedhavyas left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

teor2345 commented Jul 18, 2025

Uh oh!

teor2345 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

teor2345 left a comment

Choose a reason for hiding this comment

Uh oh!

vedhavyas commented Jul 18, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

teor2345 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

vedhavyas commented Jul 9, 2025 •

edited

Loading

vedhavyas commented Jul 14, 2025 •

edited

Loading