Skip to content
This repository has been archived by the owner on Jun 9, 2024. It is now read-only.

Marshal/Unmarshal of core Geth types is slow #446

Open
calbera opened this issue Mar 10, 2023 · 5 comments
Open

Marshal/Unmarshal of core Geth types is slow #446

calbera opened this issue Mar 10, 2023 · 5 comments
Labels
enhancement New feature or request good first issue Good for newcomers task

Comments

@calbera
Copy link
Contributor

calbera commented Mar 10, 2023

Currently, headers, transactions, and receipts are marshaled/unmarshaled using RLP encoding/decoding, which is quite slow and not specifically useful. Convert to something faster like JSON.

@calbera calbera added enhancement New feature or request good first issue Good for newcomers task labels Mar 10, 2023
@anukul
Copy link

anukul commented Mar 21, 2023

Hi! I would like to try this :D

@calbera
Copy link
Contributor Author

calbera commented Mar 21, 2023

Awesome, that would be great!

@anukul
Copy link

anukul commented Mar 22, 2023

Related - #388, will benchmark a few alternatives and proceed after review

@itsdevbear
Copy link
Member

itsdevbear commented Mar 23, 2023

mashallah @anukul

A few links here which may prove helpful:
ethereum/go-ethereum#20786
https://github.com/umbracle/fastrlp
https://github.com/vorot93/fastrlp (if there was serious performance improvements, I wouldn't be opposed to using rust, we already have CGO so using FFI is whatever)

We likely lose rlpgen if we swtich to another method, but tbh gpt4 got us covered on that one.

@anukul
Copy link

anukul commented Apr 22, 2023

Hey guys, just dropping a quick update -

fastrlp isn't always faster than rlp in my benchmarks.
I have tried to use the same approach as polygon-edge. While there is a possibility that it may not be the most optimal code, I haven't been able to find the source for the quoted benchmark.

I see that Ethereum is moving to SSZ, and it seems faster than RLP for a simpler header struct (linked above).
The go implementation does not work with geth structs out of the box yet, so I'm currently working on that on a fork (specifically this ticket) to move further.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request good first issue Good for newcomers task
Projects
None yet
Development

No branches or pull requests

3 participants