Skip to content

feat(ethexe): Implement database visitor & verifier #4787

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 43 commits into
base: master
Choose a base branch
from
Open

feat(ethexe): Implement database visitor & verifier #4787

wants to merge 43 commits into from

Conversation

ark0f
Copy link
Member

@ark0f ark0f commented Jul 23, 2025 •
edited
Loading

Closes #4624

ark0f added 24 commits July 16, 2025 19:47
@ark0f
Initial integrity verifier
f7ed370
@ark0f
integrity -> visitor
dd37fed
@ark0f
Remove unnecessary clone() on block headers
bfbba25
@ark0f
Add walk_* functions
282b295
@ark0f
Implement DatabaseVisitorStorage
4147a1b
@ark0f
Merge remote-tracking branch 'origin/master' into al/db-integrity-ver…
55de15c 
…ifier

# Conflicts:
#	ethexe/service/src/fast_sync.rs
@ark0f
Remove unnecessary clone() on block headers
087b110
@ark0f
Rename read_* and write_* storage methods for consistency
a797fec
@ark0f
Change visitor design
4ac4dcc
@ark0f
Use visitor in RequestManager
6cc8ee8
@ark0f
Walk payload lookup
e767f71
@ark0f
Convert DatabaseVisitorError into enum
9f8cadc
@ark0f
Extended verification
3281677
@ark0f
Refactor visitor to handle StateTransition and walk_block_outcome…
eab4148 
… logic
@ark0f
Extend visitor with walk_block_schedule, walk_state_transition, a…
e32dcb4 
…nd `walk_scheduled_task` logic.
@ark0f
Add comment to prevent recursion in visit_block method
5ab15fe
@ark0f
Write tests
5111374
@ark0f
Verifier tests
6e36ebc
@ark0f
Refactor visitor methods to replace walk_* functions with visit_*…
1ecf614 
… implementations and update related tests.
@ark0f
Simplify block outcome handling
db5a34e
@ark0f
Reorganize DatabaseVisitorError variants and add a comment placehol…
5d5d9d1 
…der in `verifier` module
@ark0f
Remove unused visit_* methods and fields in TestVisitor struct
0986f54
@ark0f
Check cached message queue size
5772255
@ark0f
Refactor verifier tests to use assert_eq! for error assertions
23c3958
@ark0f ark0f requested review from grishasobol and breathx July 23, 2025 17:34
@ark0f ark0f added A0-pleasereview PR is ready to be reviewed by the team D8-gearexe gear.exe-related PR labels Jul 23, 2025
@semanticdiff-com SemanticDiff.com
Copy link

semanticdiff-com bot commented Jul 23, 2025 •
edited
Loading

Review changes with  SemanticDiff

Changed Files
File Status
  ethexe/tx-pool/src/tests.rs  96% smaller
  core/src/tasks.rs  87% smaller
  ethexe/runtime/common/src/journal.rs  74% smaller
  ethexe/runtime/src/wasm/storage.rs  74% smaller
  ethexe/consensus/src/utils.rs  55% smaller
  ethexe/rpc/src/apis/block.rs  50% smaller
  ethexe/runtime/common/src/state.rs  40% smaller
  ethexe/runtime/common/src/lib.rs  33% smaller
  gtest/src/state/task_pool.rs  25% smaller
  common/numerated/src/tree.rs  17% smaller
  core/src/memory.rs  17% smaller
  core/src/program.rs  17% smaller
  ethexe/db/src/database.rs  7% smaller
  ethexe/common/src/db.rs  6% smaller
  ethexe/common/src/primitives.rs  6% smaller
  core/src/buffer.rs  1% smaller
  Cargo.lock Unsupported file format
  Cargo.toml Unsupported file format
  core/src/code/instrumented.rs  0% smaller
  core/src/code/metadata.rs  0% smaller
  core/src/pages.rs  0% smaller
  ethexe/common/Cargo.toml Unsupported file format
  ethexe/common/src/events/mirror.rs  0% smaller
  ethexe/common/src/events/mod.rs  0% smaller
  ethexe/common/src/events/router.rs  0% smaller
  ethexe/common/src/events/wvara.rs  0% smaller
  ethexe/common/src/gear.rs  0% smaller
  ethexe/compute/Cargo.toml Unsupported file format
  ethexe/compute/src/compute.rs  0% smaller
  ethexe/compute/src/prepare.rs  0% smaller
  ethexe/compute/src/service.rs  0% smaller
  ethexe/compute/src/utils.rs  0% smaller
  ethexe/consensus/src/mock.rs  0% smaller
  ethexe/db/Cargo.toml Unsupported file format
  ethexe/db/src/iterator.rs  0% smaller
  ethexe/db/src/lib.rs  0% smaller
  ethexe/db/src/verifier.rs  0% smaller
  ethexe/db/src/visitor.rs  0% smaller
  ethexe/observer/src/lib.rs  0% smaller
  ethexe/observer/src/sync.rs  0% smaller
  ethexe/processor/src/handling/run.rs  0% smaller
  ethexe/processor/src/host/threads.rs  0% smaller
  ethexe/processor/src/lib.rs  0% smaller
  ethexe/processor/src/tests.rs  0% smaller
  ethexe/rpc/src/apis/program.rs  0% smaller
  ethexe/runtime/common/Cargo.toml Unsupported file format
  ethexe/runtime/common/src/schedule.rs  0% smaller
  ethexe/runtime/src/lib.rs  0% smaller
  ethexe/runtime/src/wasm/api/instrument.rs  0% smaller
  ethexe/runtime/src/wasm/api/run.rs  0% smaller
  ethexe/service/Cargo.toml Unsupported file format
  ethexe/service/src/fast_sync.rs Unsupported file format
  ethexe/service/src/tests/mod.rs  0% smaller
  gtest/src/manager/block_exec.rs  0% smaller
  gtest/src/state/mod.rs  0% smaller
  pallets/gear/src/tests.rs  0% smaller

grishasobol
grishasobol reviewed Jul 24, 2025
View reviewed changes
Copy link
Member

@grishasobol grishasobol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cannot agree with this implementation. I think it must be done much more simplier.
Virifier - is something that implements verification logic only without walking

ethexe/db/src/verifier.rs Outdated
}

fn visit_code_valid(&mut self, _code_id: CodeId, code_valid: bool) {
if !code_valid {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why code status can not be invalid? In tests we do not check this case but this does not mean that it's impossible

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Traversal starts with chain so it should be visited only for correct codes

grishasobol
grishasobol reviewed Jul 25, 2025
View reviewed changes
Copy link
Member

@grishasobol grishasobol left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see #4793

there we do not have recursion like
walk -> visit -> walk -> visit -> walk

just
walk -> walk -> walk -> walk

@ark0f ark0f requested a review from grishasobol July 28, 2025 22:10
ark0f added 3 commits July 29, 2025 01:15
@ark0f
Clippy fixes
821926c
@ark0f
Merge remote-tracking branch 'origin/master' into al/db-integrity-ver…
61485d2 
…ifier

# Conflicts:
#	ethexe/common/src/db.rs
#	ethexe/common/src/gear.rs
#	ethexe/compute/Cargo.toml
#	ethexe/compute/src/compute.rs
#	ethexe/consensus/src/mock.rs
#	ethexe/consensus/src/utils.rs
#	ethexe/db/src/database.rs
#	ethexe/service/Cargo.toml
#	ethexe/service/src/tests/mod.rs
@ark0f
Fix tests
971713a
@ark0f ark0f force-pushed the al/db-integrity-verifier branch from 02562e4 to 971713a Compare July 30, 2025 21:54
@ecol-master
Copy link
Member

u can link issue #4624

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@breathx breathx Awaiting requested review from breathx

@grishasobol grishasobol Awaiting requested review from grishasobol

At least 1 approving review is required to merge this pull request.

Assignees

@ark0f ark0f

Labels
A0-pleasereview PR is ready to be reviewed by the team D8-gearexe gear.exe-related PR
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

ethexe: Implement DB integrity verifier
3 participants
@ark0f @ecol-master @grishasobol