-
Notifications
You must be signed in to change notification settings - Fork 11.7k
Address balances: account for storage cost of accumulator objects #24690
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
b0f5f89 to
b3c0d0a
Compare
b3c0d0a to
2aeb654
Compare
2aeb654 to
c4f36d5
Compare
c4f36d5 to
158bc6a
Compare
158bc6a to
f991234
Compare
Adds estimated cost of objects into the storage fund balance at end of epoch.
f991234 to
261c059
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds support for accounting for the storage cost of accumulator objects in the storage fund balance at the end of each epoch. The changes introduce a new end-of-epoch transaction type WriteAccumulatorStorageCost that computes and stores the estimated storage cost based on the count of accumulator objects, which is then incorporated into the storage fund balance calculation during epoch advancement. The protocol version is bumped from 105 to 106.
- Introduces
WriteAccumulatorStorageCostend-of-epoch transaction kind to record accumulator storage costs - Adds tracking of net accumulator object count through dynamic fields on the accumulator root
- Updates epoch advancement logic to include accumulator storage fund amount in balance calculations
Reviewed changes
Copilot reviewed 42 out of 44 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| crates/sui-types/src/transaction.rs | Adds WriteAccumulatorStorageCost struct and enum variant for the new transaction type |
| sui-execution/*/sui-adapter/src/execution_engine.rs | Adds panic handlers in v1/v2 engines and execution logic in latest engine |
| crates/sui-types/src/accumulator_metadata.rs | Adds functions to track and retrieve accumulator object counts |
| crates/sui-framework/packages/sui-system/sources/sui_system.move | Implements storage cost reading/writing via AccumulatorStorageCostKey in extra_fields |
| crates/sui-framework/packages/sui-system/sources/sui_system_state_inner.move | Updates advance_epoch to include accumulator storage fund amount in calculations |
| crates/sui-protocol-config/src/lib.rs | Bumps protocol version to 106 and adjusts feature flag enablement |
| crates/sui-core/src/authority.rs | Creates WriteAccumulatorStorageCost transaction at end of epoch with estimated costs |
| crates/sui-core/src/accumulators/mod.rs | Tracks object creation/deletion in barrier transactions |
| crates/sui-graphql/schema.graphql | Adds WriteAccumulatorStorageCostTransaction to GraphQL schemas |
| crates/sui-json-rpc-types/src/sui_transaction.rs | Adds enum variant for JSON-RPC representation |
| crates/sui-types/src/*_conversions.rs | Adds todo placeholders for SDK and proto conversions |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| 105 => { | ||
| cfg.feature_flags.enable_multi_epoch_transaction_expiration = true; | ||
| } | ||
| 106 => { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i don't think 105 is closed for additions yet?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it may or may not be - I have experienced wonkiness in the past when making framework changes without bumping the protocol version (sometimes it doesn't pick up the new framework if there's a snapshot for the current version already), so rather than waste time thinking about it I just bumped the version. I think there's no downside to making extra ones right?
|
Looks good overall! Could probably use a comment somewhere summarizing our discussions about why we decided to do it this way |
done |
b6b058b to
734f868
Compare
| } | ||
|
|
||
| /// Key for storing the storage cost for accumulator objects, computed at end of epoch. | ||
| public struct AccumulatorStorageCostKey has copy, drop, store {} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't need to be public?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think private structs are supported yet? if I remove public I get this error
error[E01003]: invalid modifier
┌─ /var/folders/hy/nr0t30bd76907v6q_0jfst380000gn/T/.tmp8P7tpN/sui-framework/sources/accumulator_metadata.move:147:1
│
147 │ struct AccumulatorObjectCountKey has copy, drop, store {}
│ ^^^^^^ Invalid struct declaration. Internal struct declarations are not yet supported
│
= Visibility annotations are required on struct declarations from the Move 2024 edition onwards.
let me know if I'm being stupid and there is another way to declare it non-public
| use fun accumulator_owner_destroy as Owner.destroy; | ||
|
|
||
| /// Key for storing the net count of accumulator objects as a dynamic field on the accumulator root. | ||
| public struct AccumulatorObjectCountKey has copy, drop, store {} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't need to be public
| /// Stores the computed storage cost for accumulator objects. | ||
| /// This is called by an end-of-epoch transaction to record the storage cost | ||
| /// that will be used by advance_epoch. | ||
| fun write_accumulator_storage_cost( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is the reason to use this to pass the accumulator storage cost instead of directly passing to advance_epoch?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is documented on record_accumulator_object_changes
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hmm I only see the explanation of what this function does. What I meant is that, we could add a new argument to advance_epoch function?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be fairly onerous to add a new argument to advance_epoch, IIUC
|
|
||
| /// Returns the current count of accumulator objects stored as a dynamic field. | ||
| #[allow(unused_function)] | ||
| fun get_accumulator_object_count(accumulator_root: &AccumulatorRoot): u64 { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This function doesn't seem to be used
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sorry for confusion, I'm going to add checks to the address_balance_tests that will use it but haven't finished that yet
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TIL we deprecated internal structs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok added the test changes
| Ok(Some(count)) => count, | ||
| Ok(None) => return None, | ||
| Err(e) => { | ||
| debug_fatal!("failed to read accumulator object count: {}", e); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not sure it is safe to not panic here if this errors
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it should be okay because either all validators will fail for the same reason, and then this will be skipped by all (consequently leading to the same value as last epoch being used for the storage fund adjustment), or else the local validator will fork at end-of-epoch and crash anyway? @mystenmark what do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What are the reasons this may fail?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the only possible failures would come from derive_dynamic_field_id. possibly I'm missing something but I think that can only fail if bcs::to_bytes fails, so... unlikely
| cfg.feature_flags.enable_multi_epoch_transaction_expiration = true; | ||
| } | ||
| 106 => { | ||
| // est. 100 bytes per object * 76 (storage_gas_price) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What if storage_gas_price changes?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If storage_gas_price changes, then we could either change this or leave it the same - the numbers here are merely an approximation anyways
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But we could also just use storage_gas_price directly instead of hardcoded 76?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Possibly, but if we are changing storage_gas_price we probably want to re-evaluate this again anyway (it'd be part of a much larger gas change I'd imagine?)
no existing protocol config parameters reference other parameters, so I was loath to create that dependency here
Adds estimated cost of objects into the storage fund balance at end of epoch.
Release notes
Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required.
For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates.