Skip to content

feat(state): new state package + state clean cache #2864

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 129 commits into
base: main
Choose a base branch
from

Conversation

MaksymMalicki
Copy link
Contributor

@MaksymMalicki MaksymMalicki commented Jun 2, 2025

This PR introduces a new state package, which should replace the state.go and state_snapshot.go files with their tests files. This package should be used throughout the project, the changes will appear in the following PR(s). The most important newonces:

  • Utilizing the new DB interfaces, throughout the long-lived StateDB object, with long-lived TrieDB as one of the fields. This will allow us to entirely get rid of IndexedBatch in the following PRs, which negatively affects the performance of the node with its slow writes.
  • Utilizing the clean, in-memory stateCache, which should speed up the reads. The clean cache was designed as a simple, "linked-list" like structure, which should mimic the layertree approac, without it's complex newonces (i. e. we don't need a dirty cache, since we intend to write a new state on every block). The cache handles both state Update and Revert. The clean cache also allows us to get rid of IndexedBatch, since the performance benefit of IndexBatch (quick reads from the in-memory cache layer) will be omitted. This allows us to use regular Batch with much quicker write time.
  • The commit process of the state tries is done concurrently
  • Updating the TrieDB and flushing the state changes is done concurrently

Copy link

codecov bot commented Jun 9, 2025

Codecov Report

Attention: Patch coverage is 72.26355% with 261 lines in your changes missing coverage. Please review.

Project coverage is 74.63%. Comparing base (2e3f926) to head (67252b5).

Files with missing lines Patch % Lines
core/state/state.go 64.25% 122 Missing and 56 partials ⚠️
core/state/object.go 61.19% 22 Missing and 4 partials ⚠️
core/state/history.go 66.12% 14 Missing and 7 partials ⚠️
core/state/accessors.go 87.95% 7 Missing and 3 partials ⚠️
core/trie2/triedb/hashdb/database.go 66.66% 5 Missing and 2 partials ⚠️
core/class.go 64.70% 4 Missing and 2 partials ⚠️
core/trie2/triedb/database.go 61.53% 4 Missing and 1 partial ⚠️
core/state/cache.go 96.05% 2 Missing and 1 partial ⚠️
core/trie2/trienode/node_enc.go 66.66% 2 Missing and 1 partial ⚠️
core/trie2/triedb/pathdb/cache.go 66.66% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2864      +/-   ##
==========================================
+ Coverage   74.35%   74.63%   +0.27%     
==========================================
  Files         226      233       +7     
  Lines       24688    25585     +897     
==========================================
+ Hits        18357    19095     +738     
- Misses       5105     5197      +92     
- Partials     1226     1293      +67     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@MaksymMalicki MaksymMalicki marked this pull request as ready for review June 9, 2025 09:30

type stateCache struct {
diffs map[felt.Felt]*diffCache // state root -> state diff
links map[felt.Felt]felt.Felt // child -> parent
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Am I getting this correctly, does this links represents a tree of the form

ch1 -> ch2 -> ch3 -> parent, where ch1 is child of ch2 and so on?

If that's the case I strongly believe a map is not the right data structure here. It looks that a linked list with two pointers might be a better fit for the job. If I got it correctly, then I think the best is to implement our own, didn't find any convincing package.

The pros from this is that we can now do all the cache modification in constant times, the cons is that all getter methods need to iterate through the list. To solve this, then we can add another map to the mix to be (key, ptr to linked listnode). Then we have O(1) (instead of O(128)) op for pushing/popping a layer and O(log(128)) for getter operations (stays the same).

The space cost is 128 additional pointers (64 bits each) to the current implementation. I think we are fine.

Let me know what you think.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I was thinking about the linked list as well, I like the idea. This was supposed to mock a simpler version of the layertree. I will be implementing it.

Comment on lines 76 to 83
return &State{
initRoot: stateRoot,
db: db,
contractTrie: contractTrie,
classTrie: classTrie,
stateObjects: make(map[felt.Felt]*stateObject),
}, nil
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a type that contains reference to other types, why are we returning it by reference and not by value?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants