Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implement sequencing #2332

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

implement sequencing #2332

wants to merge 1 commit into from

Conversation

rianhughes
Copy link
Contributor

@rianhughes rianhughes commented Dec 17, 2024

This PR can be broken down as follows:

  • mempool pkg: this pkg is responsible for storing incoming transactions in a persistent (so users don't have to resubmit their txns if there's a crash) linked list. Transactions come into the mempool from the AddTransaction rpc handler, and leave when builder calls Pop on the mempool.
  • genesis pkg: this pkg is essentially responsible for constructing a state-diff that can be used to bootstrap the network at block 0. For example, users can specify the set of classes they want to be declared (eg accounts / tokens), to deploy instances of those classes, and then to invoke contract functions (eg transfer funds) defined in the deployed contracts. We also allow users to specify transactions to be executed when building this state-diff. Currently, genesis_prefund_accounts.json will declare & deploy a token contract, pre-funded account contracts and the universal-deplorer contract. This allows the Juno sequencer to be used as a devnet.
  • builder pkg: this is where all the sequencing magic happens. The main entry point is Run. There are two modes of operation, "sequencer" and "shadow" mode. The normal "sequencer" mode just continuously pops transactions from the mempool (if they are present), executes them, and builds the block from the resulting traces. Every N seconds, the builder will stop executing transactions, and Finalise the block. The purpose of the "shadow" mode is to verify that the Juno sequencer can replicate the Sepolia network via re-execution. It pulls Sepolia blocks from the feeder-gateway, re-executes them, builds the traces, and verifies that there is an exact match between the sequenced block and the Sepolia block before adding the block to the chain. Note that shadowing is very sensitive to the block version, and blockifier version, and was only tested on 0.12.3 blocks.
  • other notes: there are a bunch of validatior and printer functions used to help debug errors when shadowing Sepolia. They are invaluable when trying to figure out why re-executing a Sepolia block failed to generate the expected result when "shaowing".

This PR will be broken down as follows:

  • I'll create a new PR which just contains the changes for the mempool pkg
  • We review and merge that PR
  • I'll rebase the sequencer branch
  • I'll create a new PR which just contains the changes for the genesis pkg
  • We review and merge that PR
  • I'll rebase the sequencer branch
  • At this point we can either break it down further, or just review what's left (basically just the builder pkg at this point)

@rianhughes rianhughes force-pushed the rianhughes/sequencer branch 2 times, most recently from e9ccfc0 to f91febd Compare December 17, 2024 10:02
@rianhughes rianhughes changed the title implement sequecning implement sequencing Dec 17, 2024
@NethermindEth NethermindEth deleted a comment from codecov bot Dec 17, 2024
Copy link

codecov bot commented Dec 17, 2024

Codecov Report

Attention: Patch coverage is 46.39738% with 982 lines in your changes missing coverage. Please review.

Project coverage is 70.53%. Comparing base (a9d11d4) to head (8f5f4b6).
Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
builder/builder.go 52.55% 230 Missing and 58 partials ⚠️
blockchain/blockchain.go 34.03% 90 Missing and 36 partials ⚠️
core/receipt.go 1.57% 125 Missing ⚠️
core/printers.go 0.00% 88 Missing ⚠️
core/state_update.go 43.91% 74 Missing and 9 partials ⚠️
genesis/genesis.go 64.94% 42 Missing and 19 partials ⚠️
mempool/mempool.go 63.76% 35 Missing and 15 partials ⚠️
adapters/vm2core/vm2core.go 58.40% 44 Missing and 3 partials ⚠️
node/node.go 51.13% 36 Missing and 7 partials ⚠️
node/genesis.go 0.00% 23 Missing ⚠️
... and 6 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2332      +/-   ##
==========================================
- Coverage   74.41%   70.53%   -3.88%     
==========================================
  Files         109      116       +7     
  Lines       11493    13280    +1787     
==========================================
+ Hits         8552     9367     +815     
- Misses       2277     3091     +814     
- Partials      664      822     +158     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@@ -6,6 +6,7 @@ build/
coverage/*
config/
.envrc
.python-version
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is .python-version ?

Comment on lines +38 to +44
func New(poolDB db.DB) *Pool {
return &Pool{
db: poolDB,
validator: func(_ *BroadcastedTransaction) error { return nil },
txPushed: make(chan struct{}, 1),
}
}
Copy link
Contributor

@rodrigo-pino rodrigo-pino Dec 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an example of what we talked in the daily regarding escape analysis. When returning a reference to Pool you are allocating it into the the heap, if you change the result to Pool it will be stored in the stack.

You can see the escape analysis made by the compiler using this command

go build -gcflags="-m" ./mempool/mempool.go

Which will show a line like this in the output:

...
... some compiler analysis
...

mempool/mempool.go:39:9: &Pool{...} escapes to heap

...
... other variables that escape or not
...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants