-
-
Notifications
You must be signed in to change notification settings - Fork 405
refactor: make the db interface consistent #8238
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: unstable
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Summary of Changes
Hello @nazarhussain, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request refactors the database interface to enhance consistency and cleanliness, particularly for operations where data is retrieved and then re-inserted. The changes streamline how key-value pairs are handled across different database interactions, improving code clarity and maintainability.
Highlights
- Standardized Database Interface: Introduced a new IdValue interface ({ id: I, value: V }) in abstractPrefixedRepository.ts to provide a consistent type definition for key-value pairs used in database operations.
- Refactored putManyBinary Method: The putManyBinary method in abstractPrefixedRepository.ts was updated to accept and process arrays of the new IdValue type, replacing the previous KeyValue type.
- Improved Data Migration Logic: The migrateDataColumnSidecarsFromHotToColdDb function in archiveBlocks.ts was refactored for better readability and consistency. This included extracting the data column window check into a helper function and adapting the call to putManyBinary to align with the new IdValue interface.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command>
or @gemini-code-assist <command>
. Below is a summary of the supported commands.
Feature | Command | Description |
---|---|---|
Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/
folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request refactors the database interface to be more consistent, primarily by replacing KeyValue
with a new IdValue
interface. This simplifies code in places where data is fetched and then stored again, as seen in migrateDataColumnSidecarsFromHotToColdDb
. The changes look good and improve code clarity. I've added one suggestion to further improve performance in archiveBlocks.ts
by parallelizing database reads.
for (const block of canonicalBlocks) { | ||
const blockSlot = block.slot; | ||
const blockEpoch = computeEpochAtSlot(blockSlot); | ||
|
||
if ( | ||
config.getForkSeq(blockSlot) < ForkSeq.fulu || | ||
// if block is out of ${config.MIN_EPOCHS_FOR_DATA_COLUMN_SIDECARS_REQUESTS}, skip this step | ||
blockEpoch < currentEpoch - config.MIN_EPOCHS_FOR_DATA_COLUMN_SIDECARS_REQUESTS | ||
) { | ||
continue; | ||
} | ||
if (!withinDataColumnWindow(block.slot)) continue; | ||
|
||
const dataColumnSidecarBytes = await db.dataColumnSidecar.valuesBinary(block.root); | ||
if (!dataColumnSidecarBytes) { | ||
const dataColumnSidecars = await db.dataColumnSidecar.valuesBinary(block.root); | ||
if (!dataColumnSidecars || dataColumnSidecars.length === 0) { | ||
throw Error(`No dataColumnSidecars found for slot ${block.slot} root ${toHex(block.root)}`); | ||
} | ||
promises.push( | ||
db.dataColumnSidecarArchive.putManyBinary( | ||
block.slot, | ||
dataColumnSidecarBytes.map((p) => ({key: p.id, value: p.value})) | ||
) | ||
); | ||
migratedWrappedDataColumns += dataColumnSidecarBytes.length; | ||
|
||
promises.push(db.dataColumnSidecarArchive.putManyBinary(block.slot, dataColumnSidecars)); | ||
migratedDataColumnsCount += dataColumnSidecars.length; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While the refactoring has simplified the code, there's still a performance improvement that can be made here. The await
inside the for...of
loop for db.dataColumnSidecar.valuesBinary()
causes the database calls to be executed sequentially for each block in the batch. These calls can be parallelized using Promise.all
to improve performance, especially for large batches.
const blocksToProcess = canonicalBlocks.filter((block) => withinDataColumnWindow(block.slot));
const sidecarsByBlock = await Promise.all(
blocksToProcess.map((block) => db.dataColumnSidecar.valuesBinary(block.root))
);
for (let i = 0; i < blocksToProcess.length; i++) {
const block = blocksToProcess[i];
const dataColumnSidecars = sidecarsByBlock[i];
if (!dataColumnSidecars || dataColumnSidecars.length === 0) {
throw Error(`No dataColumnSidecars found for slot ${block.slot} root ${toHex(block.root)}`);
}
promises.push(db.dataColumnSidecarArchive.putManyBinary(block.slot, dataColumnSidecars));
migratedDataColumnsCount += dataColumnSidecars.length;
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Iterating over all canonicalBlocks
once for filter
and then for data column checks will make it less performant. So why I intionally used the continue
statement with single loop.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## unstable #8238 +/- ##
============================================
- Coverage 54.25% 54.17% -0.09%
============================================
Files 846 848 +2
Lines 63799 63900 +101
Branches 4832 4846 +14
============================================
+ Hits 34617 34618 +1
- Misses 29106 29205 +99
- Partials 76 77 +1 🚀 New features to boost your workflow:
|
|
Benchmark suite | Current: 76e53e4 | Previous: 9348949 | Ratio |
---|---|---|---|
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 | 94.711 us/op | 24.665 us/op | 3.84 |
Array.fill - length 1000000 | 11.329 ms/op | 2.8947 ms/op | 3.91 |
Array push - length 1000000 | 51.166 ms/op | 14.076 ms/op | 3.63 |
Full benchmark results
Benchmark suite | Current: 76e53e4 | Previous: 9348949 | Ratio |
---|---|---|---|
getPubkeys - index2pubkey - req 1000 vs - 250000 vc | 1.0646 ms/op | 741.08 us/op | 1.44 |
getPubkeys - validatorsArr - req 1000 vs - 250000 vc | 37.772 us/op | 28.936 us/op | 1.31 |
BLS verify - blst | 884.40 us/op | 864.84 us/op | 1.02 |
BLS verifyMultipleSignatures 3 - blst | 1.2186 ms/op | 1.3130 ms/op | 0.93 |
BLS verifyMultipleSignatures 8 - blst | 1.8447 ms/op | 2.1392 ms/op | 0.86 |
BLS verifyMultipleSignatures 32 - blst | 5.0838 ms/op | 4.4947 ms/op | 1.13 |
BLS verifyMultipleSignatures 64 - blst | 9.3608 ms/op | 8.3384 ms/op | 1.12 |
BLS verifyMultipleSignatures 128 - blst | 18.027 ms/op | 16.023 ms/op | 1.13 |
BLS deserializing 10000 signatures | 706.85 ms/op | 621.12 ms/op | 1.14 |
BLS deserializing 100000 signatures | 7.1577 s/op | 6.2300 s/op | 1.15 |
BLS verifyMultipleSignatures - same message - 3 - blst | 990.07 us/op | 963.53 us/op | 1.03 |
BLS verifyMultipleSignatures - same message - 8 - blst | 1.1292 ms/op | 1.0950 ms/op | 1.03 |
BLS verifyMultipleSignatures - same message - 32 - blst | 1.7978 ms/op | 1.6969 ms/op | 1.06 |
BLS verifyMultipleSignatures - same message - 64 - blst | 3.0434 ms/op | 2.5773 ms/op | 1.18 |
BLS verifyMultipleSignatures - same message - 128 - blst | 4.8227 ms/op | 4.2012 ms/op | 1.15 |
BLS aggregatePubkeys 32 - blst | 20.128 us/op | 17.742 us/op | 1.13 |
BLS aggregatePubkeys 128 - blst | 72.558 us/op | 62.903 us/op | 1.15 |
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 | 57.464 ms/op | 38.635 ms/op | 1.49 |
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 | 54.284 ms/op | 36.520 ms/op | 1.49 |
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 | 41.304 ms/op | 39.554 ms/op | 1.04 |
getSlashingsAndExits - default max | 82.907 us/op | 50.612 us/op | 1.64 |
getSlashingsAndExits - 2k | 376.31 us/op | 295.07 us/op | 1.28 |
proposeBlockBody type=full, size=empty | 8.2294 ms/op | 5.0711 ms/op | 1.62 |
isKnown best case - 1 super set check | 218.00 ns/op | 408.00 ns/op | 0.53 |
isKnown normal case - 2 super set checks | 212.00 ns/op | 406.00 ns/op | 0.52 |
isKnown worse case - 16 super set checks | 213.00 ns/op | 406.00 ns/op | 0.52 |
InMemoryCheckpointStateCache - add get delete | 2.5520 us/op | 2.4300 us/op | 1.05 |
validate api signedAggregateAndProof - struct | 1.5415 ms/op | 1.9552 ms/op | 0.79 |
validate gossip signedAggregateAndProof - struct | 1.5357 ms/op | 1.4426 ms/op | 1.06 |
batch validate gossip attestation - vc 640000 - chunk 32 | 119.19 us/op | 108.86 us/op | 1.09 |
batch validate gossip attestation - vc 640000 - chunk 64 | 103.85 us/op | 93.407 us/op | 1.11 |
batch validate gossip attestation - vc 640000 - chunk 128 | 98.094 us/op | 87.124 us/op | 1.13 |
batch validate gossip attestation - vc 640000 - chunk 256 | 100.55 us/op | 86.139 us/op | 1.17 |
pickEth1Vote - no votes | 988.86 us/op | 797.41 us/op | 1.24 |
pickEth1Vote - max votes | 5.6602 ms/op | 4.1644 ms/op | 1.36 |
pickEth1Vote - Eth1Data hashTreeRoot value x2048 | 12.191 ms/op | 8.4548 ms/op | 1.44 |
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 | 15.064 ms/op | 15.152 ms/op | 0.99 |
pickEth1Vote - Eth1Data fastSerialize value x2048 | 467.16 us/op | 352.42 us/op | 1.33 |
pickEth1Vote - Eth1Data fastSerialize tree x2048 | 2.0988 ms/op | 1.9443 ms/op | 1.08 |
bytes32 toHexString | 370.00 ns/op | 517.00 ns/op | 0.72 |
bytes32 Buffer.toString(hex) | 248.00 ns/op | 410.00 ns/op | 0.60 |
bytes32 Buffer.toString(hex) from Uint8Array | 332.00 ns/op | 501.00 ns/op | 0.66 |
bytes32 Buffer.toString(hex) + 0x | 250.00 ns/op | 433.00 ns/op | 0.58 |
Object access 1 prop | 0.12000 ns/op | 0.31000 ns/op | 0.39 |
Map access 1 prop | 0.13000 ns/op | 0.31800 ns/op | 0.41 |
Object get x1000 | 6.1310 ns/op | 5.1310 ns/op | 1.19 |
Map get x1000 | 6.3060 ns/op | 5.8670 ns/op | 1.07 |
Object set x1000 | 29.137 ns/op | 18.952 ns/op | 1.54 |
Map set x1000 | 19.790 ns/op | 17.246 ns/op | 1.15 |
Return object 10000 times | 0.29100 ns/op | 0.28630 ns/op | 1.02 |
Throw Error 10000 times | 4.4661 us/op | 3.6243 us/op | 1.23 |
toHex | 141.45 ns/op | 98.705 ns/op | 1.43 |
Buffer.from | 138.15 ns/op | 95.846 ns/op | 1.44 |
shared Buffer | 86.474 ns/op | 69.645 ns/op | 1.24 |
fastMsgIdFn sha256 / 200 bytes | 2.1720 us/op | 1.9280 us/op | 1.13 |
fastMsgIdFn h32 xxhash / 200 bytes | 198.00 ns/op | 381.00 ns/op | 0.52 |
fastMsgIdFn h64 xxhash / 200 bytes | 254.00 ns/op | 428.00 ns/op | 0.59 |
fastMsgIdFn sha256 / 1000 bytes | 7.1490 us/op | 5.7400 us/op | 1.25 |
fastMsgIdFn h32 xxhash / 1000 bytes | 337.00 ns/op | 507.00 ns/op | 0.66 |
fastMsgIdFn h64 xxhash / 1000 bytes | 332.00 ns/op | 502.00 ns/op | 0.66 |
fastMsgIdFn sha256 / 10000 bytes | 65.121 us/op | 48.641 us/op | 1.34 |
fastMsgIdFn h32 xxhash / 10000 bytes | 1.8300 us/op | 1.8490 us/op | 0.99 |
fastMsgIdFn h64 xxhash / 10000 bytes | 1.1980 us/op | 1.2990 us/op | 0.92 |
send data - 1000 256B messages | 15.069 ms/op | 15.582 ms/op | 0.97 |
send data - 1000 512B messages | 19.250 ms/op | 18.684 ms/op | 1.03 |
send data - 1000 1024B messages | 26.682 ms/op | 24.546 ms/op | 1.09 |
send data - 1000 1200B messages | 23.252 ms/op | 20.616 ms/op | 1.13 |
send data - 1000 2048B messages | 23.514 ms/op | 19.627 ms/op | 1.20 |
send data - 1000 4096B messages | 26.823 ms/op | 17.773 ms/op | 1.51 |
send data - 1000 16384B messages | 42.910 ms/op | 32.461 ms/op | 1.32 |
send data - 1000 65536B messages | 115.32 ms/op | 77.617 ms/op | 1.49 |
enrSubnets - fastDeserialize 64 bits | 910.00 ns/op | 945.00 ns/op | 0.96 |
enrSubnets - ssz BitVector 64 bits | 322.00 ns/op | 492.00 ns/op | 0.65 |
enrSubnets - fastDeserialize 4 bits | 130.00 ns/op | 313.00 ns/op | 0.42 |
enrSubnets - ssz BitVector 4 bits | 323.00 ns/op | 497.00 ns/op | 0.65 |
prioritizePeers score -10:0 att 32-0.1 sync 2-0 | 234.70 us/op | 198.83 us/op | 1.18 |
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 | 264.60 us/op | 227.38 us/op | 1.16 |
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 | 370.76 us/op | 364.68 us/op | 1.02 |
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 | 699.65 us/op | 638.65 us/op | 1.10 |
prioritizePeers score 0:0 att 64-1 sync 4-1 | 845.78 us/op | 733.06 us/op | 1.15 |
array of 16000 items push then shift | 1.6239 us/op | 1.2828 us/op | 1.27 |
LinkedList of 16000 items push then shift | 7.0500 ns/op | 6.3230 ns/op | 1.11 |
array of 16000 items push then pop | 89.989 ns/op | 62.039 ns/op | 1.45 |
LinkedList of 16000 items push then pop | 6.8760 ns/op | 6.1290 ns/op | 1.12 |
array of 24000 items push then shift | 2.4133 us/op | 1.8781 us/op | 1.29 |
LinkedList of 24000 items push then shift | 7.0100 ns/op | 6.2270 ns/op | 1.13 |
array of 24000 items push then pop | 124.83 ns/op | 86.576 ns/op | 1.44 |
LinkedList of 24000 items push then pop | 6.8400 ns/op | 6.1280 ns/op | 1.12 |
intersect bitArray bitLen 8 | 6.3220 ns/op | 5.4190 ns/op | 1.17 |
intersect array and set length 8 | 37.891 ns/op | 32.030 ns/op | 1.18 |
intersect bitArray bitLen 128 | 29.584 ns/op | 26.347 ns/op | 1.12 |
intersect array and set length 128 | 625.22 ns/op | 538.81 ns/op | 1.16 |
bitArray.getTrueBitIndexes() bitLen 128 | 1.0210 us/op | 1.1220 us/op | 0.91 |
bitArray.getTrueBitIndexes() bitLen 248 | 1.8170 us/op | 1.7660 us/op | 1.03 |
bitArray.getTrueBitIndexes() bitLen 512 | 3.7430 us/op | 3.2490 us/op | 1.15 |
Buffer.concat 32 items | 598.00 ns/op | 753.00 ns/op | 0.79 |
Uint8Array.set 32 items | 939.00 ns/op | 944.00 ns/op | 0.99 |
Buffer.copy | 2.1220 us/op | 1.9330 us/op | 1.10 |
Uint8Array.set - with subarray | 1.9370 us/op | 1.5190 us/op | 1.28 |
Uint8Array.set - without subarray | 1.5260 us/op | 903.00 ns/op | 1.69 |
getUint32 - dataview | 203.00 ns/op | 374.00 ns/op | 0.54 |
getUint32 - manual | 122.00 ns/op | 307.00 ns/op | 0.40 |
Set add up to 64 items then delete first | 2.3774 us/op | 1.8195 us/op | 1.31 |
OrderedSet add up to 64 items then delete first | 3.4127 us/op | 2.8273 us/op | 1.21 |
Set add up to 64 items then delete last | 2.3740 us/op | 1.8832 us/op | 1.26 |
OrderedSet add up to 64 items then delete last | 3.7504 us/op | 3.1374 us/op | 1.20 |
Set add up to 64 items then delete middle | 2.5483 us/op | 1.8834 us/op | 1.35 |
OrderedSet add up to 64 items then delete middle | 5.7442 us/op | 4.5205 us/op | 1.27 |
Set add up to 128 items then delete first | 5.3522 us/op | 4.2523 us/op | 1.26 |
OrderedSet add up to 128 items then delete first | 8.1442 us/op | 6.4764 us/op | 1.26 |
Set add up to 128 items then delete last | 5.1105 us/op | 3.9670 us/op | 1.29 |
OrderedSet add up to 128 items then delete last | 7.5579 us/op | 6.1138 us/op | 1.24 |
Set add up to 128 items then delete middle | 4.8848 us/op | 3.9628 us/op | 1.23 |
OrderedSet add up to 128 items then delete middle | 13.743 us/op | 11.635 us/op | 1.18 |
Set add up to 256 items then delete first | 10.538 us/op | 8.0736 us/op | 1.31 |
OrderedSet add up to 256 items then delete first | 16.068 us/op | 12.990 us/op | 1.24 |
Set add up to 256 items then delete last | 9.9881 us/op | 8.0411 us/op | 1.24 |
OrderedSet add up to 256 items then delete last | 14.884 us/op | 14.465 us/op | 1.03 |
Set add up to 256 items then delete middle | 9.9627 us/op | 7.8564 us/op | 1.27 |
OrderedSet add up to 256 items then delete middle | 41.781 us/op | 36.304 us/op | 1.15 |
transfer serialized Status (84 B) | 2.2180 us/op | 1.9670 us/op | 1.13 |
copy serialized Status (84 B) | 1.1770 us/op | 1.2020 us/op | 0.98 |
transfer serialized SignedVoluntaryExit (112 B) | 2.2180 us/op | 1.9040 us/op | 1.16 |
copy serialized SignedVoluntaryExit (112 B) | 1.1810 us/op | 1.1970 us/op | 0.99 |
transfer serialized ProposerSlashing (416 B) | 2.2660 us/op | 1.9780 us/op | 1.15 |
copy serialized ProposerSlashing (416 B) | 1.2730 us/op | 1.3070 us/op | 0.97 |
transfer serialized Attestation (485 B) | 2.2540 us/op | 1.9470 us/op | 1.16 |
copy serialized Attestation (485 B) | 1.2780 us/op | 1.3320 us/op | 0.96 |
transfer serialized AttesterSlashing (33232 B) | 2.8220 us/op | 1.9480 us/op | 1.45 |
copy serialized AttesterSlashing (33232 B) | 4.4410 us/op | 2.4750 us/op | 1.79 |
transfer serialized Small SignedBeaconBlock (128000 B) | 3.4310 us/op | 1.9660 us/op | 1.75 |
copy serialized Small SignedBeaconBlock (128000 B) | 10.277 us/op | 5.4740 us/op | 1.88 |
transfer serialized Avg SignedBeaconBlock (200000 B) | 3.9650 us/op | 2.1310 us/op | 1.86 |
copy serialized Avg SignedBeaconBlock (200000 B) | 17.004 us/op | 8.1190 us/op | 2.09 |
transfer serialized BlobsSidecar (524380 B) | 4.2590 us/op | 2.6920 us/op | 1.58 |
copy serialized BlobsSidecar (524380 B) | 70.948 us/op | 119.25 us/op | 0.59 |
transfer serialized Big SignedBeaconBlock (1000000 B) | 3.9150 us/op | 3.2650 us/op | 1.20 |
copy serialized Big SignedBeaconBlock (1000000 B) | 161.40 us/op | 113.84 us/op | 1.42 |
pass gossip attestations to forkchoice per slot | 2.8200 ms/op | 2.4013 ms/op | 1.17 |
forkChoice updateHead vc 100000 bc 64 eq 0 | 457.78 us/op | 376.44 us/op | 1.22 |
forkChoice updateHead vc 600000 bc 64 eq 0 | 2.8048 ms/op | 2.3271 ms/op | 1.21 |
forkChoice updateHead vc 1000000 bc 64 eq 0 | 4.7893 ms/op | 3.8790 ms/op | 1.23 |
forkChoice updateHead vc 600000 bc 320 eq 0 | 2.9036 ms/op | 2.2828 ms/op | 1.27 |
forkChoice updateHead vc 600000 bc 1200 eq 0 | 2.8639 ms/op | 2.3857 ms/op | 1.20 |
forkChoice updateHead vc 600000 bc 7200 eq 0 | 3.0789 ms/op | 2.4636 ms/op | 1.25 |
forkChoice updateHead vc 600000 bc 64 eq 1000 | 10.430 ms/op | 9.5449 ms/op | 1.09 |
forkChoice updateHead vc 600000 bc 64 eq 10000 | 10.338 ms/op | 9.3487 ms/op | 1.11 |
forkChoice updateHead vc 600000 bc 64 eq 300000 | 13.470 ms/op | 11.458 ms/op | 1.18 |
computeDeltas 500000 validators 300 proto nodes | 3.9834 ms/op | 3.1191 ms/op | 1.28 |
computeDeltas 500000 validators 1200 proto nodes | 4.1314 ms/op | 3.2553 ms/op | 1.27 |
computeDeltas 500000 validators 7200 proto nodes | 4.0322 ms/op | 3.1885 ms/op | 1.26 |
computeDeltas 750000 validators 300 proto nodes | 5.9526 ms/op | 4.6953 ms/op | 1.27 |
computeDeltas 750000 validators 1200 proto nodes | 6.0566 ms/op | 4.8127 ms/op | 1.26 |
computeDeltas 750000 validators 7200 proto nodes | 6.2922 ms/op | 5.0018 ms/op | 1.26 |
computeDeltas 1400000 validators 300 proto nodes | 13.447 ms/op | 9.3094 ms/op | 1.44 |
computeDeltas 1400000 validators 1200 proto nodes | 12.289 ms/op | 9.0583 ms/op | 1.36 |
computeDeltas 1400000 validators 7200 proto nodes | 11.264 ms/op | 9.7261 ms/op | 1.16 |
computeDeltas 2100000 validators 300 proto nodes | 16.888 ms/op | 14.238 ms/op | 1.19 |
computeDeltas 2100000 validators 1200 proto nodes | 18.594 ms/op | 13.740 ms/op | 1.35 |
computeDeltas 2100000 validators 7200 proto nodes | 20.566 ms/op | 14.149 ms/op | 1.45 |
altair processAttestation - 250000 vs - 7PWei normalcase | 2.8763 ms/op | 2.9091 ms/op | 0.99 |
altair processAttestation - 250000 vs - 7PWei worstcase | 4.0362 ms/op | 3.5766 ms/op | 1.13 |
altair processAttestation - setStatus - 1/6 committees join | 150.75 us/op | 106.47 us/op | 1.42 |
altair processAttestation - setStatus - 1/3 committees join | 313.07 us/op | 210.61 us/op | 1.49 |
altair processAttestation - setStatus - 1/2 committees join | 408.19 us/op | 314.89 us/op | 1.30 |
altair processAttestation - setStatus - 2/3 committees join | 499.70 us/op | 386.48 us/op | 1.29 |
altair processAttestation - setStatus - 4/5 committees join | 712.30 us/op | 560.80 us/op | 1.27 |
altair processAttestation - setStatus - 100% committees join | 815.10 us/op | 632.74 us/op | 1.29 |
altair processBlock - 250000 vs - 7PWei normalcase | 5.2737 ms/op | 4.6499 ms/op | 1.13 |
altair processBlock - 250000 vs - 7PWei normalcase hashState | 31.371 ms/op | 31.658 ms/op | 0.99 |
altair processBlock - 250000 vs - 7PWei worstcase | 43.775 ms/op | 44.344 ms/op | 0.99 |
altair processBlock - 250000 vs - 7PWei worstcase hashState | 96.799 ms/op | 79.641 ms/op | 1.22 |
phase0 processBlock - 250000 vs - 7PWei normalcase | 2.0477 ms/op | 2.0393 ms/op | 1.00 |
phase0 processBlock - 250000 vs - 7PWei worstcase | 28.223 ms/op | 22.022 ms/op | 1.28 |
altair processEth1Data - 250000 vs - 7PWei normalcase | 374.31 us/op | 280.21 us/op | 1.34 |
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 | 11.201 us/op | 9.3140 us/op | 1.20 |
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 | 94.711 us/op | 24.665 us/op | 3.84 |
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 | 25.833 us/op | 12.368 us/op | 2.09 |
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 | 13.080 us/op | 6.6960 us/op | 1.95 |
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 | 200.94 us/op | 171.63 us/op | 1.17 |
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 | 2.0381 ms/op | 1.4918 ms/op | 1.37 |
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 | 2.8607 ms/op | 1.9281 ms/op | 1.48 |
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 | 2.7360 ms/op | 1.9439 ms/op | 1.41 |
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 | 6.7383 ms/op | 4.0975 ms/op | 1.64 |
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 | 2.9495 ms/op | 1.9941 ms/op | 1.48 |
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 | 7.3241 ms/op | 4.0342 ms/op | 1.82 |
Tree 40 250000 create | 803.54 ms/op | 396.44 ms/op | 2.03 |
Tree 40 250000 get(125000) | 182.85 ns/op | 107.46 ns/op | 1.70 |
Tree 40 250000 set(125000) | 3.1904 us/op | 1.3422 us/op | 2.38 |
Tree 40 250000 toArray() | 49.221 ms/op | 15.002 ms/op | 3.28 |
Tree 40 250000 iterate all - toArray() + loop | 36.467 ms/op | 13.822 ms/op | 2.64 |
Tree 40 250000 iterate all - get(i) | 82.008 ms/op | 47.726 ms/op | 1.72 |
Array 250000 create | 4.8563 ms/op | 2.5519 ms/op | 1.90 |
Array 250000 clone - spread | 6.0587 ms/op | 679.78 us/op | 8.91 |
Array 250000 get(125000) | 0.84900 ns/op | 0.75300 ns/op | 1.13 |
Array 250000 set(125000) | 1.0560 ns/op | 0.59800 ns/op | 1.77 |
Array 250000 iterate all - loop | 107.27 us/op | 127.12 us/op | 0.84 |
phase0 afterProcessEpoch - 250000 vs - 7PWei | 46.527 ms/op | 40.012 ms/op | 1.16 |
Array.fill - length 1000000 | 11.329 ms/op | 2.8947 ms/op | 3.91 |
Array push - length 1000000 | 51.166 ms/op | 14.076 ms/op | 3.63 |
Array.get | 0.39695 ns/op | 0.33066 ns/op | 1.20 |
Uint8Array.get | 0.52599 ns/op | 0.41590 ns/op | 1.26 |
phase0 beforeProcessEpoch - 250000 vs - 7PWei | 38.937 ms/op | 19.617 ms/op | 1.98 |
altair processEpoch - mainnet_e81889 | 328.88 ms/op | 267.15 ms/op | 1.23 |
mainnet_e81889 - altair beforeProcessEpoch | 24.275 ms/op | 18.464 ms/op | 1.31 |
mainnet_e81889 - altair processJustificationAndFinalization | 5.9970 us/op | 4.6590 us/op | 1.29 |
mainnet_e81889 - altair processInactivityUpdates | 6.0167 ms/op | 3.5941 ms/op | 1.67 |
mainnet_e81889 - altair processRewardsAndPenalties | 40.998 ms/op | 27.537 ms/op | 1.49 |
mainnet_e81889 - altair processRegistryUpdates | 794.00 ns/op | 953.00 ns/op | 0.83 |
mainnet_e81889 - altair processSlashings | 209.00 ns/op | 406.00 ns/op | 0.51 |
mainnet_e81889 - altair processEth1DataReset | 186.00 ns/op | 394.00 ns/op | 0.47 |
mainnet_e81889 - altair processEffectiveBalanceUpdates | 1.3259 ms/op | 1.1706 ms/op | 1.13 |
mainnet_e81889 - altair processSlashingsReset | 931.00 ns/op | 1.1900 us/op | 0.78 |
mainnet_e81889 - altair processRandaoMixesReset | 1.2200 us/op | 1.4100 us/op | 0.87 |
mainnet_e81889 - altair processHistoricalRootsUpdate | 187.00 ns/op | 413.00 ns/op | 0.45 |
mainnet_e81889 - altair processParticipationFlagUpdates | 575.00 ns/op | 885.00 ns/op | 0.65 |
mainnet_e81889 - altair processSyncCommitteeUpdates | 145.00 ns/op | 361.00 ns/op | 0.40 |
mainnet_e81889 - altair afterProcessEpoch | 47.927 ms/op | 42.450 ms/op | 1.13 |
capella processEpoch - mainnet_e217614 | 1.1486 s/op | 860.61 ms/op | 1.33 |
mainnet_e217614 - capella beforeProcessEpoch | 71.943 ms/op | 69.679 ms/op | 1.03 |
mainnet_e217614 - capella processJustificationAndFinalization | 5.8440 us/op | 4.7410 us/op | 1.23 |
mainnet_e217614 - capella processInactivityUpdates | 16.736 ms/op | 12.465 ms/op | 1.34 |
mainnet_e217614 - capella processRewardsAndPenalties | 194.64 ms/op | 194.12 ms/op | 1.00 |
mainnet_e217614 - capella processRegistryUpdates | 7.3110 us/op | 6.9050 us/op | 1.06 |
mainnet_e217614 - capella processSlashings | 221.00 ns/op | 491.00 ns/op | 0.45 |
mainnet_e217614 - capella processEth1DataReset | 193.00 ns/op | 548.00 ns/op | 0.35 |
mainnet_e217614 - capella processEffectiveBalanceUpdates | 4.6235 ms/op | 3.7844 ms/op | 1.22 |
mainnet_e217614 - capella processSlashingsReset | 962.00 ns/op | 1.1540 us/op | 0.83 |
mainnet_e217614 - capella processRandaoMixesReset | 1.2420 us/op | 1.5880 us/op | 0.78 |
mainnet_e217614 - capella processHistoricalRootsUpdate | 239.00 ns/op | 471.00 ns/op | 0.51 |
mainnet_e217614 - capella processParticipationFlagUpdates | 568.00 ns/op | 745.00 ns/op | 0.76 |
mainnet_e217614 - capella afterProcessEpoch | 133.15 ms/op | 114.21 ms/op | 1.17 |
phase0 processEpoch - mainnet_e58758 | 321.81 ms/op | 337.13 ms/op | 0.95 |
mainnet_e58758 - phase0 beforeProcessEpoch | 83.419 ms/op | 89.357 ms/op | 0.93 |
mainnet_e58758 - phase0 processJustificationAndFinalization | 6.0840 us/op | 4.9750 us/op | 1.22 |
mainnet_e58758 - phase0 processRewardsAndPenalties | 45.711 ms/op | 43.421 ms/op | 1.05 |
mainnet_e58758 - phase0 processRegistryUpdates | 3.4070 us/op | 2.8750 us/op | 1.19 |
mainnet_e58758 - phase0 processSlashings | 189.00 ns/op | 406.00 ns/op | 0.47 |
mainnet_e58758 - phase0 processEth1DataReset | 187.00 ns/op | 458.00 ns/op | 0.41 |
mainnet_e58758 - phase0 processEffectiveBalanceUpdates | 1.6852 ms/op | 1.0153 ms/op | 1.66 |
mainnet_e58758 - phase0 processSlashingsReset | 1.0340 us/op | 1.3180 us/op | 0.78 |
mainnet_e58758 - phase0 processRandaoMixesReset | 1.2620 us/op | 1.6940 us/op | 0.74 |
mainnet_e58758 - phase0 processHistoricalRootsUpdate | 192.00 ns/op | 475.00 ns/op | 0.40 |
mainnet_e58758 - phase0 processParticipationRecordUpdates | 1.2060 us/op | 1.4930 us/op | 0.81 |
mainnet_e58758 - phase0 afterProcessEpoch | 37.943 ms/op | 34.572 ms/op | 1.10 |
phase0 processEffectiveBalanceUpdates - 250000 normalcase | 1.5431 ms/op | 1.9800 ms/op | 0.78 |
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 | 4.0549 ms/op | 1.4988 ms/op | 2.71 |
altair processInactivityUpdates - 250000 normalcase | 20.793 ms/op | 21.152 ms/op | 0.98 |
altair processInactivityUpdates - 250000 worstcase | 19.756 ms/op | 21.255 ms/op | 0.93 |
phase0 processRegistryUpdates - 250000 normalcase | 12.142 us/op | 8.7490 us/op | 1.39 |
phase0 processRegistryUpdates - 250000 badcase_full_deposits | 330.48 us/op | 317.15 us/op | 1.04 |
phase0 processRegistryUpdates - 250000 worstcase 0.5 | 150.10 ms/op | 104.07 ms/op | 1.44 |
altair processRewardsAndPenalties - 250000 normalcase | 41.673 ms/op | 25.466 ms/op | 1.64 |
altair processRewardsAndPenalties - 250000 worstcase | 38.564 ms/op | 27.811 ms/op | 1.39 |
phase0 getAttestationDeltas - 250000 normalcase | 16.653 ms/op | 7.1096 ms/op | 2.34 |
phase0 getAttestationDeltas - 250000 worstcase | 10.710 ms/op | 13.923 ms/op | 0.77 |
phase0 processSlashings - 250000 worstcase | 139.44 us/op | 99.425 us/op | 1.40 |
altair processSyncCommitteeUpdates - 250000 | 14.404 ms/op | 10.046 ms/op | 1.43 |
BeaconState.hashTreeRoot - No change | 327.00 ns/op | 450.00 ns/op | 0.73 |
BeaconState.hashTreeRoot - 1 full validator | 140.96 us/op | 71.728 us/op | 1.97 |
BeaconState.hashTreeRoot - 32 full validator | 1.9339 ms/op | 642.94 us/op | 3.01 |
BeaconState.hashTreeRoot - 512 full validator | 18.125 ms/op | 9.7443 ms/op | 1.86 |
BeaconState.hashTreeRoot - 1 validator.effectiveBalance | 165.34 us/op | 98.601 us/op | 1.68 |
BeaconState.hashTreeRoot - 32 validator.effectiveBalance | 2.1484 ms/op | 1.7811 ms/op | 1.21 |
BeaconState.hashTreeRoot - 512 validator.effectiveBalance | 40.147 ms/op | 25.462 ms/op | 1.58 |
BeaconState.hashTreeRoot - 1 balances | 108.56 us/op | 73.524 us/op | 1.48 |
BeaconState.hashTreeRoot - 32 balances | 1.0824 ms/op | 662.27 us/op | 1.63 |
BeaconState.hashTreeRoot - 512 balances | 11.127 ms/op | 7.4256 ms/op | 1.50 |
BeaconState.hashTreeRoot - 250000 balances | 233.28 ms/op | 217.09 ms/op | 1.07 |
aggregationBits - 2048 els - zipIndexesInBitList | 35.952 us/op | 20.686 us/op | 1.74 |
byteArrayEquals 32 | 59.075 ns/op | 44.031 ns/op | 1.34 |
Buffer.compare 32 | 19.044 ns/op | 15.765 ns/op | 1.21 |
byteArrayEquals 1024 | 1.8295 us/op | 1.2867 us/op | 1.42 |
Buffer.compare 1024 | 34.462 ns/op | 24.329 ns/op | 1.42 |
byteArrayEquals 16384 | 29.479 us/op | 20.421 us/op | 1.44 |
Buffer.compare 16384 | 227.87 ns/op | 194.38 ns/op | 1.17 |
byteArrayEquals 123687377 | 213.38 ms/op | 154.95 ms/op | 1.38 |
Buffer.compare 123687377 | 11.005 ms/op | 4.9138 ms/op | 2.24 |
byteArrayEquals 32 - diff last byte | 58.567 ns/op | 42.972 ns/op | 1.36 |
Buffer.compare 32 - diff last byte | 18.762 ns/op | 16.209 ns/op | 1.16 |
byteArrayEquals 1024 - diff last byte | 1.7723 us/op | 1.2898 us/op | 1.37 |
Buffer.compare 1024 - diff last byte | 28.087 ns/op | 24.512 ns/op | 1.15 |
byteArrayEquals 16384 - diff last byte | 28.215 us/op | 20.429 us/op | 1.38 |
Buffer.compare 16384 - diff last byte | 211.29 ns/op | 170.21 ns/op | 1.24 |
byteArrayEquals 123687377 - diff last byte | 204.48 ms/op | 156.18 ms/op | 1.31 |
Buffer.compare 123687377 - diff last byte | 7.1705 ms/op | 6.4526 ms/op | 1.11 |
byteArrayEquals 32 - random bytes | 5.3260 ns/op | 5.7450 ns/op | 0.93 |
Buffer.compare 32 - random bytes | 18.141 ns/op | 57.454 ns/op | 0.32 |
byteArrayEquals 1024 - random bytes | 5.3320 ns/op | 4.7560 ns/op | 1.12 |
Buffer.compare 1024 - random bytes | 18.401 ns/op | 15.290 ns/op | 1.20 |
byteArrayEquals 16384 - random bytes | 5.3740 ns/op | 4.7570 ns/op | 1.13 |
Buffer.compare 16384 - random bytes | 18.070 ns/op | 15.176 ns/op | 1.19 |
byteArrayEquals 123687377 - random bytes | 6.5800 ns/op | 7.5000 ns/op | 0.88 |
Buffer.compare 123687377 - random bytes | 18.830 ns/op | 17.980 ns/op | 1.05 |
regular array get 100000 times | 45.351 us/op | 39.859 us/op | 1.14 |
wrappedArray get 100000 times | 45.462 us/op | 39.947 us/op | 1.14 |
arrayWithProxy get 100000 times | 12.595 ms/op | 9.0464 ms/op | 1.39 |
ssz.Root.equals | 47.392 ns/op | 39.132 ns/op | 1.21 |
byteArrayEquals | 46.515 ns/op | 39.009 ns/op | 1.19 |
Buffer.compare | 10.595 ns/op | 8.9100 ns/op | 1.19 |
processSlot - 1 slots | 10.692 us/op | 8.8950 us/op | 1.20 |
processSlot - 32 slots | 3.0771 ms/op | 2.3979 ms/op | 1.28 |
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei | 3.1419 ms/op | 2.2361 ms/op | 1.41 |
getCommitteeAssignments - req 1 vs - 250000 vc | 2.1949 ms/op | 1.8511 ms/op | 1.19 |
getCommitteeAssignments - req 100 vs - 250000 vc | 4.2514 ms/op | 3.6488 ms/op | 1.17 |
getCommitteeAssignments - req 1000 vs - 250000 vc | 4.5261 ms/op | 3.8735 ms/op | 1.17 |
findModifiedValidators - 10000 modified validators | 823.02 ms/op | 754.23 ms/op | 1.09 |
findModifiedValidators - 1000 modified validators | 801.02 ms/op | 654.13 ms/op | 1.22 |
findModifiedValidators - 100 modified validators | 325.27 ms/op | 184.64 ms/op | 1.76 |
findModifiedValidators - 10 modified validators | 182.08 ms/op | 126.28 ms/op | 1.44 |
findModifiedValidators - 1 modified validators | 194.84 ms/op | 132.58 ms/op | 1.47 |
findModifiedValidators - no difference | 289.11 ms/op | 141.68 ms/op | 2.04 |
compare ViewDUs | 6.3597 s/op | 6.1888 s/op | 1.03 |
compare each validator Uint8Array | 1.5960 s/op | 1.9156 s/op | 0.83 |
compare ViewDU to Uint8Array | 1.5524 s/op | 811.44 ms/op | 1.91 |
migrate state 1000000 validators, 24 modified, 0 new | 897.79 ms/op | 841.29 ms/op | 1.07 |
migrate state 1000000 validators, 1700 modified, 1000 new | 1.2395 s/op | 1.1777 s/op | 1.05 |
migrate state 1000000 validators, 3400 modified, 2000 new | 1.3987 s/op | 1.4180 s/op | 0.99 |
migrate state 1500000 validators, 24 modified, 0 new | 957.12 ms/op | 866.39 ms/op | 1.10 |
migrate state 1500000 validators, 1700 modified, 1000 new | 1.2095 s/op | 1.1103 s/op | 1.09 |
migrate state 1500000 validators, 3400 modified, 2000 new | 1.4236 s/op | 1.3168 s/op | 1.08 |
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei | 4.2500 ns/op | 5.6800 ns/op | 0.75 |
state getBlockRootAtSlot - 250000 vs - 7PWei | 553.61 ns/op | 485.18 ns/op | 1.14 |
naive computeProposerIndex 100000 validators | 54.690 ms/op | 42.393 ms/op | 1.29 |
computeProposerIndex 100000 validators | 1.5240 ms/op | 1.3125 ms/op | 1.16 |
naiveGetNextSyncCommitteeIndices 1000 validators | 7.9702 s/op | 6.7149 s/op | 1.19 |
getNextSyncCommitteeIndices 1000 validators | 122.30 ms/op | 98.208 ms/op | 1.25 |
naiveGetNextSyncCommitteeIndices 10000 validators | 7.9171 s/op | 6.2976 s/op | 1.26 |
getNextSyncCommitteeIndices 10000 validators | 112.04 ms/op | 97.273 ms/op | 1.15 |
naiveGetNextSyncCommitteeIndices 100000 validators | 8.2402 s/op | 7.1778 s/op | 1.15 |
getNextSyncCommitteeIndices 100000 validators | 119.20 ms/op | 95.415 ms/op | 1.25 |
naive computeShuffledIndex 100000 validators | 25.243 s/op | 25.119 s/op | 1.00 |
cached computeShuffledIndex 100000 validators | 565.38 ms/op | 485.33 ms/op | 1.16 |
naive computeShuffledIndex 2000000 validators | 513.05 s/op | 459.59 s/op | 1.12 |
cached computeShuffledIndex 2000000 validators | 37.178 s/op | 25.840 s/op | 1.44 |
computeProposers - vc 250000 | 615.66 us/op | 559.37 us/op | 1.10 |
computeEpochShuffling - vc 250000 | 41.798 ms/op | 39.589 ms/op | 1.06 |
getNextSyncCommittee - vc 250000 | 10.685 ms/op | 9.5785 ms/op | 1.12 |
computeSigningRoot for AttestationData | 21.108 us/op | 16.704 us/op | 1.26 |
hash AttestationData serialized data then Buffer.toString(base64) | 1.5878 us/op | 1.1768 us/op | 1.35 |
toHexString serialized data | 1.2140 us/op | 949.74 ns/op | 1.28 |
Buffer.toString(base64) | 155.53 ns/op | 107.26 ns/op | 1.45 |
nodejs block root to RootHex using toHex | 146.68 ns/op | 109.98 ns/op | 1.33 |
nodejs block root to RootHex using toRootHex | 91.914 ns/op | 76.791 ns/op | 1.20 |
nodejs fromhex(blob) | 115.46 ms/op | 84.781 ms/op | 1.36 |
nodejs fromHexInto(blob) | 97.044 ms/op | 81.235 ms/op | 1.19 |
browser block root to RootHex using the deprecated toHexString | 209.86 ns/op | 186.63 ns/op | 1.12 |
browser block root to RootHex using toHex | 175.19 ns/op | 150.75 ns/op | 1.16 |
browser block root to RootHex using toRootHex | 158.39 ns/op | 139.06 ns/op | 1.14 |
browser fromHexInto(blob) | 815.15 us/op | 644.16 us/op | 1.27 |
browser fromHex(blob) | 792.69 ms/op | 619.38 ms/op | 1.28 |
by benchmarkbot/action
packages/beacon-node/src/chain/archiveStore/utils/archiveBlocks.ts
Outdated
Show resolved
Hide resolved
merge conflicts |
Motivation
Make the DB interface more cleaner and consistent.
Description
Steps to test or reproduce