Skip to content

Only require nodes to publish custody columns from reconstruction #4657

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Only require nodes to publish custody columns from reconstruction #4657

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
40f6e5e
Nodes to only publish only custody columns from reconstruction.
jimmygchen Oct 15, 2025
fb8a2b0
Update wording
jimmygchen Oct 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 8 additions & 9 deletions specs/fulu/das-core.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -244,21 +244,20 @@ columns from other peers.

## Reconstruction and cross-seeding

If the node obtains 50%+ of all the columns, it SHOULD reconstruct the full data
matrix via the `recover_matrix` helper. Nodes MAY delay this reconstruction
allowing time for other columns to arrive over the network. If delaying
reconstruction, nodes may use a random delay in order to desynchronize
If the node obtains more than 50% of all the columns, it SHOULD reconstruct the
Copy link
Contributor

@cskiraly cskiraly Oct 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually this should be "at least 50%"

Copy link
Contributor Author

@jimmygchen jimmygchen Oct 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing this out, I actually meant to say

If the node custodies more than 50% of columns, and has obtained at least 50% of all columns, it SHOULD reconstruct...

I'll push an update

Copy link
Contributor Author

@jimmygchen jimmygchen Oct 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in fb8a2b0

Copy link
Member

@jtraglia jtraglia Oct 16, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jimmygchen this is still technically incorrect. It should be:

If the node custodies at least 50% of columns

We do not need to custody 51% of columns to do this. We just need 50%.

Edit: Oh wait. I see what you're saying now. Please ignore my original comment. What you have written here makes perfect sense. There's no point in doing reconstruction if your node custodies exactly 50% of columns as there's nothing to gain.

Copy link
Contributor

@cskiraly cskiraly Oct 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, there is. There is nothing to gain for the node from the custody perspective, but:

  • it will have the blobs (columns 0-63, i.e. the systematic columns), which might be a gain for the node itself
  • it can contribute to the diffusion, which is kind of nice from the system perspective

Copy link
Member

@jtraglia jtraglia Oct 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

which might be a gain for the node itself

But very few nodes need this. If that's important, one could just override the node's custody to be columns 0-63.

it can contribute to the diffusion

Yes, but is this really necessary given that there will be supernodes doing this?

Copy link
Contributor Author

@jimmygchen jimmygchen Oct 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.. one could just override the node's custody to be columns 0-63.

Agree with @jtraglia on most points but I think we may want to avoid doing the above , which may lead to more saturation in subnets 0-63 on the network.

Ideally clients support either semi-supernode or RPC for blob retrievals.

full data matrix via the `recover_matrix` helper. Nodes MAY delay this
reconstruction allowing time for other columns to arrive over the network. If
delaying reconstruction, nodes may use a random delay in order to desynchronize
reconstruction among nodes, thus reducing overall CPU load.

Once the node obtains a column through reconstruction, the node MUST expose the
new column as if it had received it over the network. If the node is subscribed
to the subnet corresponding to the column, it MUST send the reconstructed
`DataColumnSidecar` to its topic mesh neighbors. If instead the node is not
subscribed to the corresponding subnet, it SHOULD still expose the availability
of the `DataColumnSidecar` as part of the gossip emission process. After
exposing the reconstructed `DataColumnSidecar` to the network, the node MAY
delete the `DataColumnSidecar` if it is not part of the node's custody
requirement.
subscribed to the corresponding subnet, it MAY still expose the availability of
the `DataColumnSidecar` as part of the gossip emission process. After exposing
the reconstructed `DataColumnSidecar` to the network, the node MAY delete the
`DataColumnSidecar` if it is not part of the node's custody requirement.

*Note*: A node always maintains a matrix view of the rows and columns they are
following, able to cross-reference and cross-seed in either direction.
Expand Down