refactor!: change wire protocol to use uni streams per topic #75
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Documentation for this PR has been generated and is available at: https://n0-computer.github.io/iroh-gossip/pr/75/docs/iroh_gossip/ Last updated: 2025-06-09T11:08:56Z |
Simulation reportLast updated: 2025-06-09T11:09:57Z |
Frando
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Wire-breaking change
Based on #67
Changes the wire protocol to use uni streams per topic instead of a single bidirectional stream for all topics. This saves 32 bytes per message because we don't have to include the topic id in each message anymore.
Peers open uni streams whenever they want to send a message for a topic for which no uni stream is open yet. First, a header is written. The header is length-prefixed and versioned, so that we can do non-breaking upgrades here. Right now, the header only contains the topic id.
Peers accept uni streams, read the header, and then forward messages to the proto state machine, as previously.
Right now the recv code path does not check if the topic is open (this will need further refactoring) but this is, in a way, no change to the status quo, were peers could also send you messages for unknown topics (they will simply be ignored). But with this change, we can later optimize to close accepted streams for unjoined topics.
Breaking Changes
/iroh-gossip/1. There's no downwards compatibility.net::utilmodule is now private. The utilities in here already were not intended to be used in other crates. If you were using something from there, copy-paste it into your crate.Notes & open questions
For now, the structure of the net module is mostly unchanged. Sending is still fully sequential, even though we could send on multiple topics in parallel now. I have an unfinished branch that has a much bigger refactoring of the
netmodule to use actor loops per topic instead of a global actor loop. However this is still unfinished, and I wanted to get a smaller version with only the wire-protocol changes out first. The refactoring to use task loops per topic is then purely internal without further wire breakage.Change checklist