Fix and refactor trie proof logics #2252

weiihann · 2024-11-05T09:16:27Z

This PR primarily does the following:

Refactor VerifyProof as it fails on some edge cases
Refactor VerifyRangeProof to cover most edge cases

A new data structure OrderedSet is used for proof generation and verification. The previous implementation used slices to contain proof nodes, which introduced complexity, unnecessary duplication, and inefficiency. The OrderedSet structure acts as a key-value database, simplifying the proof logic and improving readability.

VerifyRangeProof is not fully completed. For some specific missing key in the range (i.e. gapped range), the function would deem the edge case as valid but it's not supposed to be. The root cause still needs further investigation, but it's most likely due to the Trie structure and how we perform the root hash calculation. A separate PR will be raised to tackle it.

Credits: some implementation details are based on go-ethereum

codecov · 2024-11-05T09:26:40Z

Codecov Report

Attention: Patch coverage is 66.85934% with 172 lines in your changes missing coverage. Please review.

Project coverage is 74.56%. Comparing base (2b1b219) to head (ed7d0f5).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
core/trie/proof.go	65.48%	78 Missing and 29 partials ⚠️
core/trie/trie.go	56.04%	34 Missing and 6 partials ⚠️
core/trie/node.go	51.42%	12 Missing and 5 partials ⚠️
core/trie/key.go	82.22%	7 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2252      +/-   ##
==========================================
- Coverage   75.24%   74.56%   -0.68%     
==========================================
  Files         108      109       +1     
  Lines       11520    11559      +39     
==========================================
- Hits         8668     8619      -49     
- Misses       2185     2270      +85     
- Partials      667      670       +3

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

pnowosie

1st part of review

core/trie/key.go

core/trie/proof.go

core/trie/trie.go

pnowosie

Nice work! 👍

core/trie/proof.go

core/trie/node.go

core/trie/proof.go

utils/orderedset.go

core/trie/key.go

core/trie/node.go

core/trie/trie.go

core/trie/proof_test.go

rianhughes · 2024-12-10T14:12:48Z

Very nice work :)

verify proof passes all test cases range proof but has issues prove non-membership add non-membership tests doesnt work some fixes more tests doesn't work fully pass 4 key trie D pass trie c and d pass all 4 keys tidy up tests fix shiftright bug pass 9 keys non-existent proof to path pass all test cases tidy up tests storageNodeSet's Put will merge with existing one

Co-authored-by: Pawel Nowosielski <[email protected]> Signed-off-by: Ng Wei Han <[email protected]>

kirugan · 2024-12-12T10:21:50Z

utils/orderedset.go

+	// Insert new entry
+	ps.itemPos[key] = len(ps.items)
+	ps.items = append(ps.items, value)
+	ps.size++


why do we need size? can we use len(ps.items) instead ?

commit 7687360 Author: Ng Wei Han <[email protected]> Date: Thu Dec 12 18:54:32 2024 +0800 Remove size in OrderedSet (#2319) commit 65b7507 Author: Ng Wei Han <[email protected]> Date: Thu Dec 12 18:20:55 2024 +0800 Fix and refactor trie proof logics (#2252) commit 2b1b219 Author: aleven1999 <[email protected]> Date: Thu Dec 12 12:11:28 2024 +0400 Remove unused code (#2318) commit 0a21162 Author: Daniil Ankushin <[email protected]> Date: Thu Dec 12 00:04:08 2024 +0700 Remove unused code (#2317) commit 8bf9be9 Author: Rian Hughes <[email protected]> Date: Wed Dec 11 14:11:22 2024 +0200 update invoke v3 txn validation to require sender_address (#2308) commit 91d0f8e Author: Kirill <[email protected]> Date: Wed Dec 11 16:01:10 2024 +0400 Add schema_version to output of db info command (#2309) commit 60e8cc9 Author: AnavarKh <[email protected]> Date: Wed Dec 11 16:04:31 2024 +0530 Update download link for Juno snapshots from dev to io in Readme file (#2314) commit 8862de1 Author: wojciechos <[email protected]> Date: Wed Dec 11 11:20:02 2024 +0100 Improve binary build workflow for cross-platform releases (#2315) - Add proper architecture handling in matrix configuration - Implement caching for Go modules and Rust dependencies - Streamline dependency installation for both Linux and macOS - Improve binary artifact handling and checksums - Add retention policy for build artifacts - Split build steps for better clarity and maintainability This update ensures more reliable and efficient binary builds across all supported platforms. commit e75e504 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed Dec 11 07:35:16 2024 +0000 Bump nanoid from 3.3.7 to 3.3.8 in /docs in the npm_and_yarn group across 1 directory (#2316) Bump nanoid in /docs in the npm_and_yarn group across 1 directory Bumps the npm_and_yarn group with 1 update in the /docs directory: [nanoid](https://github.com/ai/nanoid). Updates `nanoid` from 3.3.7 to 3.3.8 - [Release notes](https://github.com/ai/nanoid/releases) - [Changelog](https://github.com/ai/nanoid/blob/main/CHANGELOG.md) - [Commits](ai/nanoid@3.3.7...3.3.8) --- updated-dependencies: - dependency-name: nanoid dependency-type: indirect dependency-group: npm_and_yarn ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 3a7abeb Author: wojciechos <[email protected]> Date: Tue Dec 10 21:52:49 2024 +0100 Skip error logs for FGW responses with NOT_RECEIVED status (#2303) * Add NotReceived case handling in adaptTransactionStatus --------- Co-authored-by: Rian Hughes <[email protected]>

pnowosie mentioned this pull request Nov 5, 2024

Starknet's getStorageProof rpc method #2194

Open

weiihann force-pushed the weiihann/refactor-proof branch 2 times, most recently from 44f9ae6 to 59f2edd Compare December 2, 2024 12:11

weiihann marked this pull request as ready for review December 2, 2024 12:11

weiihann changed the title ~~Fixes get proof and verify proof~~ Fix and refactor trie proof logics Dec 2, 2024

pnowosie self-requested a review December 3, 2024 13:00

pnowosie reviewed Dec 5, 2024

View reviewed changes

weiihann force-pushed the weiihann/refactor-proof branch from 9d37fdf to d1b26ec Compare December 5, 2024 09:19

pnowosie self-requested a review December 6, 2024 11:18

pnowosie approved these changes Dec 6, 2024

View reviewed changes

core/trie/proof.go Show resolved Hide resolved

core/trie/node.go Outdated Show resolved Hide resolved

core/trie/proof.go Show resolved Hide resolved

core/trie/proof.go Outdated Show resolved Hide resolved

utils/orderedset.go Outdated Show resolved Hide resolved

weiihann force-pushed the weiihann/refactor-proof branch from 4dc6386 to 2ea3e78 Compare December 9, 2024 12:03