The `chia.consensus` module no longer depends upon `chia.full_node` #19671

richardkiss · 2025-06-03T20:12:01Z

Prior to this change, there were several dependencies of chia.full_node in the chia.consensus module.

…eck_time_locks`.

coveralls-official · 2025-06-03T21:32:49Z

Pull Request Test Coverage Report for Build 15426877319

Details

224 of 261 (85.82%) changed or added relevant lines in 31 files are covered.
17 unchanged lines in 6 files lost coverage.
Overall coverage increased (+0.009%) to 91.229%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
chia/consensus/block_height_map_abc.py	21	28	75.0%
chia/consensus/coin_store_abc.py	31	41	75.61%
chia/consensus/block_store_abc.py	52	72	72.22%

Files with Coverage Reduction	New Missed Lines	%
chia/daemon/client.py	1	74.72%
chia/server/node_discovery.py	1	81.93%
chia/full_node/full_node.py	2	88.23%
chia/util/files.py	2	91.38%
chia/rpc/rpc_server.py	3	88.54%
chia/daemon/server.py	8	83.46%

Totals
Change from base Build 15426193337:	0.009%
Covered Lines:	101506
Relevant Lines:	111148

💛 - Coveralls

arvidn · 2025-06-04T07:14:02Z

is there a vision of what chia.consensus should contain? because "consensus rules" are spread out virtually everywhere today

arvidn

I think it's reasonable to move and rename mempool_check_time_locks().
I don't see the rationale for moving BlockHeightMap out of Blockchain, it seems like you just make it more complicated to construct the Blockchain object, and I don't see an upside. We don't need the BlockHeightMap outside Blockchain, do we?
Also, I'm under the impression that we prefer Protocols over inheritance (I personally do at least). We have some precedence of using protocols as well.
However, at a higher level, why do you need an interface/protocol for these types? The main reason I can think if is if you want to be able to stub them out in tests, but I don't see that. It seems like added complexity with no upside, but perhaps I'm missing it.

arvidn · 2025-06-04T07:23:15Z

chia/full_node/block_height_map.py

it seems a lot simpler to just move this class into chia.consensus and use concrete types. We only have one class implementing the interface, and it doesn't look like we have any intention of adding more implementations, so we should just use the concrete type and keep things simple, I think

The reasoning is I want to keep storage details away from consensus. I actually think chia.consensus does a pretty good job of isolating the consensus stuff (not including mempool, which IMO isn't actually consensus. I prefer to narrowly define consensus as "blockchain validation"). This opens the door to (for example) creating a completely RAM-based consensus mechanism that just stores all the data in python dictionaries.

We could move the three storage classes into consensus. If you look at the implementations of BlockHeightMap, CoinStore and BlockStore, they are, IMO, ugly and messy and bogged-down in SQLite3 administrivia that distracts from the understanding of how consensus works. I'd actually quite like to move DBWrapper and friends etc. into a new module called chia.sqlite3 or chia.db, to isolate the storage implementations into a black box.

Note that I am working on a new implementation of CoinStore right now (the rocks_db thing) and having an ABC will make it easier to split CoinStore into ConsensusCoinStore (or whatever) and ExplorerStore (indices of parent coin info and puzzle hashes) which I'm sure you've heard me harp on about several times now.

I think it's a bit funny that python supports ABC and Protocol. It makes it hard to know which one to use. ChatGPT recommended ABC for this case. I guess Protocol is useful for built-in types like saying "I need a thing that looks like a bytes object in the following ways...". For CoinStore and friends, I kind of prefer ABC because it allows default implementations (for example, get_coin_record could easily have a default imp that uses get_coin_records with very little loss in efficiency), and it makes it easier to determine exactly what you have to write and what is still missing without resorting to mypy (which for me always produces a lot of noise that signal gets lost in because I'm probably doing something wrong).

An argument for avoiding SQLite3 code in consensus: it's not side-effect free. It's much easier to reason about pure functional code that has no side effects. The ABC is side-effect free (well, except I guess it mutates the state of the object, which I guess is generally seen as okay). Any disk-based stuff is loaded with side effects.

Additionally, this change will make it easier to provide a transition path where we temporarily support the SQLite3 DB and the RocksDB for CoinStore (and if we move more stuff to rocksdb in future, it helps there too).

ABC is ancient. Protocol is more recent. i encourage protocols since they don't use inheritance and whatever related to that.

richardkiss · 2025-06-04T18:39:13Z

chia/consensus/coin_store_abc.py

+    This is a substitute for importing from chia.full_node.coin_store directly.
+    """
+
+    db_wrapper: DBWrapper2


This line should be removed for sure.

richardkiss · 2025-06-04T18:40:04Z

chia/consensus/coin_store_abc.py

+    This is a substitute for importing from chia.full_node.coin_store directly.
+    """
+
+    db_wrapper: DBWrapper2


Suggested change

db_wrapper: DBWrapper2

richardkiss · 2025-06-05T21:02:02Z

chia/full_node/block_store.py

+        """Provide a transaction context."""
+
+        async with self.db_wrapper.writer() as transaction:
+            yield transaction


Please check this part carefully. I think it's right, but I got a weird transient error on one test set: py3.10 Mac OS Intel simulator, something about trying to operate on a closed DB. I don't recall having seen that before so it's possible it's related to this.

richardkiss · 2025-06-05T21:02:28Z

chia/full_node/block_store.py

-                for row in await cursor.fetchall():
-                    block_rec = BlockRecord.from_bytes(row[1])
-                    all_blocks[block_rec.header_hash] = block_rec
+        batch_size = self.db_wrapper.host_parameter_limit


Batching moved down into here.

richardkiss · 2025-06-05T21:02:49Z

chia/full_node/block_store.py

-                raise ValueError(f"Header hash {hh} not in the blockchain")
-            ret.append(all_blocks[hh])
-        return ret
+        # Step 1: Check the cache for each hash


caching moved down into here

richardkiss · 2025-06-05T21:21:08Z

This is getting a bit too complicated. I think that I will withdraw it and submit multiple PRs that are smaller and simpler.

github-actions · 2025-06-12T18:15:18Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

richardkiss added 5 commits June 3, 2025 13:07

chia/full_node/signage_point.py -> chia/consensus/signage_point.py

decca11

Move mempool_check_time_locks to chia.consensus and rename to `ch…

0c1f3db

…eck_time_locks`.

Create BlockHeightMapABC.

784417a

[Block|Coin]StoreABC.

3e67127

The chia.protocols module no longer depends upon chia.full_node

10db231

richardkiss requested a review from arvidn June 3, 2025 20:12

richardkiss requested a review from a team as a code owner June 3, 2025 20:12

richardkiss added Changed Required label for PR that categorizes merge commit message as "Changed" for changelog Exclude_Notes Use this label if the changes in the PR should be excluded from the release notes labels Jun 3, 2025

richardkiss requested a review from altendky June 3, 2025 20:12

aqk previously approved these changes Jun 3, 2025

View reviewed changes

arvidn reviewed Jun 4, 2025

View reviewed changes

richardkiss commented Jun 4, 2025

View reviewed changes

Use Protocol instead of ABC.

6625f84

richardkiss dismissed aqk’s stale review via 6625f84 June 5, 2025 05:30

richardkiss force-pushed the tach branch 11 times, most recently from 37ae7f3 to 57effcc Compare June 5, 2025 19:36

richardkiss force-pushed the tach branch 3 times, most recently from 46be01a to f842243 Compare June 5, 2025 19:58

Deal with removed member variables from BlockStoreProtocol.

914a487

richardkiss force-pushed the tach branch from f842243 to 914a487 Compare June 5, 2025 20:07

richardkiss commented Jun 5, 2025

View reviewed changes

github-actions bot added the merge_conflict Branch has conflicts that prevent merge to main label Jun 12, 2025

richardkiss marked this pull request as draft June 13, 2025 17:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

The `chia.consensus` module no longer depends upon `chia.full_node` #19671

The `chia.consensus` module no longer depends upon `chia.full_node` #19671

Uh oh!

richardkiss commented Jun 3, 2025

Uh oh!

coveralls-official bot commented Jun 3, 2025

Uh oh!

arvidn commented Jun 4, 2025

Uh oh!

arvidn left a comment

Uh oh!

arvidn Jun 4, 2025

Uh oh!

richardkiss Jun 4, 2025

Uh oh!

richardkiss Jun 4, 2025

Uh oh!

richardkiss Jun 4, 2025

Uh oh!

altendky Jun 4, 2025

Uh oh!

richardkiss Jun 4, 2025

Uh oh!

richardkiss Jun 4, 2025

Uh oh!

richardkiss Jun 5, 2025

Uh oh!

richardkiss Jun 5, 2025

Uh oh!

richardkiss Jun 5, 2025

Uh oh!

richardkiss commented Jun 5, 2025

Uh oh!

github-actions bot commented Jun 12, 2025

Uh oh!

Uh oh!

The chia.consensus module no longer depends upon chia.full_node #19671

Are you sure you want to change the base?

The chia.consensus module no longer depends upon chia.full_node #19671

Uh oh!

Conversation

richardkiss commented Jun 3, 2025

Uh oh!

coveralls-official bot commented Jun 3, 2025

Pull Request Test Coverage Report for Build 15426877319

Details

💛 - Coveralls

Uh oh!

arvidn commented Jun 4, 2025

Uh oh!

arvidn left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

richardkiss commented Jun 5, 2025

Uh oh!

github-actions bot commented Jun 12, 2025

Uh oh!

Uh oh!

The `chia.consensus` module no longer depends upon `chia.full_node` #19671

The `chia.consensus` module no longer depends upon `chia.full_node` #19671