Add Cell Dissemination via Partial Message Specification

[MarcoPolo]

In draft until the libp2p/specs work is merged and we have multiple implementations of this. Also requires TODOs to be addressed.

This PR specifies how consensus clients use Gossipsub's Partial Messages to disseminate cells rather than only full columns.

More context in this ethresearch post.

[fradamt]

I think there's three possible ways to go with this:

1. Add an Optional SSZ type, which we are currently missing (see https://ethereum-magicians.org/t/eip-6475-ssz-optional/12891)

class PartialDataColumnSidecar(Container):
    index: ColumnIndex
    cells_bitmap: Bitlist[MAX_BLOB_COMMITMENTS_PER_BLOCK]
    cells: List[Cell, MAX_BLOB_COMMITMENTS_PER_BLOCK]
    kzg_proofs: List[KZGProof, MAX_BLOB_COMMITMENTS_PER_BLOCK]
    kzg_commitments: Optional[List[KZGCommitment, MAX_BLOB_COMMITMENTS_PER_BLOCK]]
    signed_block_header: Optional[SignedBeaconBlockHeader]
    kzg_commitments_inclusion_proof: Optional[Vector[Bytes32, KZG_COMMITMENTS_INCLUSION_PROOF_DEPTH]

2. Separate the data part from the kzg commitments, header and proof part

class PartialDataColumnSidecar(Container):
    index: ColumnIndex
    cells_bitmap: Bitlist[MAX_BLOB_COMMITMENTS_PER_BLOCK]
    cells: List[Cell, MAX_BLOB_COMMITMENTS_PER_BLOCK]
    kzg_proofs: List[KZGProof, MAX_BLOB_COMMITMENTS_PER_BLOCK]

class KZGCommitmentsSidecar(Container):
    kzg_commitments: List[KZGCommitment, MAX_BLOB_COMMITMENTS_PER_BLOCK]
    signed_block_header: SignedBeaconBlockHeader
    kzg_commitments_inclusion_proof: Vector[Bytes32, KZG_COMMITMENTS_INCLUSION_PROOF_DEPTH]

3. Have two different types for a column with and without committments (hopefully with nicer names)

class PartialDataColumnSidecar(Container):
    index: ColumnIndex
    cells_bitmap: Bitlist[MAX_BLOB_COMMITMENTS_PER_BLOCK]
    cells: List[Cell, MAX_BLOB_COMMITMENTS_PER_BLOCK]
    kzg_proofs: List[KZGProof, MAX_BLOB_COMMITMENTS_PER_BLOCK]

class PartialDataColumnSidecarWithCommitments(Container):
    index: ColumnIndex
    cells_bitmap: Bitlist[MAX_BLOB_COMMITMENTS_PER_BLOCK]
    cells: List[Cell, MAX_BLOB_COMMITMENTS_PER_BLOCK]
    kzg_proofs: List[KZGProof, MAX_BLOB_COMMITMENTS_PER_BLOCK]
    kzg_commitments: List[KZGCommitment, MAX_BLOB_COMMITMENTS_PER_BLOCK]
    signed_block_header: SignedBeaconBlockHeader
    kzg_commitments_inclusion_proof: Vector[Bytes32, KZG_COMMITMENTS_INCLUSION_PROOF_DEPTH]

Also:

• changed ByteVector to Bitlist (but could also be Bitvector)
• NUMBER_OF_COLUMNS isn't the length/max-length of a column, should be MAX_BLOB_COMMITMENTS_PER_BLOCK
• I'd just leave the column_index in all the time, 8 bytes doesn't seem worth worrying about imho

[MarcoPolo]

Preference for 1, as the having two containers makes things a bit more annoying as you need to then define when one or the other (or both) are transmitted.

I'd just leave the column_index in all the time, 8 bytes doesn't seem worth worrying about imho

The only reason we would need it is when the subnet != column index, right? 8 bytes is small, but a couple bytes here and a couple bytes there and soon we have a whole packet of redundant data. My preference might be to just remove this field and leave it as future work to define how to derive a column index when the subnet count != column count.

[ralexstokes]

1 may seem better but the other options move the complexity from the type system (and the requisite push to add a new concept to the ssz spec) to other layers of the stack

that said, we have have Option as a non-canonical type for some time now, and it may be time to go ahead and add formal support for it

[MarcoPolo]

Including the commitments here is an optimization. We could rely on just receiving them in the block body.

[fradamt]

Something worth paying attention to here is also how DataColumnSidedcar evolves in the gloas spec (though these changes will likely pre-date gloas). At the moment it seems like header + inclusion proof will be removed in favor of beacon_block_root only, requiring the beacon block to be seen in advance (though this still will not contain the kzg commitments), which is somewhat natural with epbs.

[fradamt]

We could do the same thing here, requiring the block (or a full sidecar) to have been seen before receiving a partial column (and excluding the commitments, since at this point they're still in the beacon block). Similar to what you did below but with beacon_block_root instead of the whole signed_beacon_block_header

class PartialDataColumnSidecar(Container):
    index: ColumnIndex
    cells_present_bitmap: Bitlist[MAX_BLOB_COMMITMENTS_PER_BLOCK]
    partial_column: List[Cell, MAX_BLOB_COMMITMENTS_PER_BLOCK]
    kzg_proofs: List[KZGProof, MAX_BLOB_COMMITMENTS_PER_BLOCK]
    beacon_block_root: Root

[fradamt]

This is what I am referring to re: gloas specs btw https://github.com/ethereum/consensus-specs/pull/4527/files

[raulk]

While this mechanistically introduces cell-level deltas, how they precisely integrate into existing PeerDAS processes isn't very clear. How about a section covering the end-to-end behaviour:

• When nodes push DataColumnSidecars
• When nodes send partial IWANTs
• When nodes send partial IHAVEs
• When nodes send PartialDataColumnSidecars
• Dependency on getBlobsV3

[AgeManning]

We've been in the process of implementing this and have run into a few issues specifically around the validation. We probably need to think about and add the validation rules to this spec.

Our current thinking is to probably include the kzg-commitments into the cell data so that we can verify each cell as we get them.

If we don't have them in there (as the current design), we have to wait for the block data before we can verify if the cells are accurate or not. This also entails adding some kind of cache to keep track of which peer sent want in order to penalize them at a future time if we find out the data is invalid.

This then opens up a few attack vectors, where peers can send us invalid data, filling the cache preventing us getting valid data etc. We would have to be careful with the cache also, to account for periods of non-finality with a high degree of forking where there can be many valid chains.

Currently it seems the complexity of handing all these edge cases might outweigh the relatively small bandwidth savings we are making by excluding the kzg commitments.

Maybe there is a clear implementation path here we are missing however.

cc @dknopik

[MarcoPolo]

We’ve been in the process of implementing this and have run into a few issues specifically around the validation. We probably need to think about and add the validation rules to this spec.

Agreed.

Our current thinking is to probably include the kzg-commitments into the cell data so that we can verify each cell as we get them.

If we don’t have them in there (as the current design), we have to wait for the block data before we can verify if the cells are accurate or not. This also entails adding some kind of cache to keep track of which peer sent want in order to penalize them at a future time if we find out the data is invalid.

One thing to point out is that this is only an issue for cells you get via an eager push. Without eager push, peers only give you cells that they know you’re missing, and giving a peer this information only happens after getting the block.

This then opens up a few attack vectors, where peers can send us invalid data, filling the cache preventing us getting valid data etc.

The cache is only relevant in the eager push case. If the cache is full, it wouldn’t prevent you from getting valid data after you’ve received you block.

We would have to be careful with the cache also, to account for periods of non-finality with a high degree of forking where there can be many valid chains.

I’m unfamiliar with this scenario. Can you explain a bit more please? Do we expect multiple valid proposers as well?

Currently it seems the complexity of handing all these edge cases might outweigh the relatively small bandwidth savings we are making by excluding the kzg commitments.

No strong opinion on what we do here overall, but I’ll point out that the gloas spec also removes some of the fields and requires the validator to wait for the builder’s bid first.

https://github.com/ethereum/consensus-specs/blob/master/specs/gloas/p2p-interface.md#modified-datacolumnsidecar.

Maybe there is a clear implementation path here we are missing however.

My plan was to limit the eager push cache to 2-3 messages per topic. On first partial publish, we can validate the cache and down score peers appropriately.

I don’t think it’s an interesting attack to poison this cache as it, worst case, costs the victim 1 RTT of delay to receive the cells, at the expense of getting down scored and pruned.

That said, if we wanted to include additional information for validating eager pushes of cells before we see a blob we could send them the relevant parts (KZGCommitments, Inclusion Proof, Signed Beacon block header) only when it’s an eager push.

@fradamt, do you have any extra context on why gloas removes those fields useful for validating the column before the block in the DataColumnSidecar?

[fradamt]

Our current thinking is to probably include the kzg-commitments into the cell data so that we can verify each cell as we get them.

Currently it seems the complexity of handing all these edge cases might outweigh the relatively small bandwidth savings we are making by excluding the kzg commitments.

I don't see how this is viable. Cells are 2 KBs (and might get down to even 1 KB in the future), so including all KZG commitments means 48 KBs * number of blobs, which is 100% overhead as soon as we get to ~40 blobs. The bandwidth saving are very large, if we care about being able to efficiently send individual cells and not just large-ish partial columns (which imo is essential).

Something else we could do is to only include the kzg_commitment of the cells we have, and an inclusion proof per cell. could include the inclusion proof of the single kzg_commitment for this cell. This is more reasonable, but still quite expensive because depth 12+ (at >= 128 blobs), so 12*32 + 48 = 432 bytes, or ~21% extra overhead, and 23% total (counting the kzg proof as well), per 2 KB cell. As it is, it's not great but might be fine, but it might prevent us from making cells even thinner, since at 1 KB cells we'd be at 46% total overhead?

class PartialDataColumnSidecar(Container):
    cells_present_bitmap: Bitlist[MAX_BLOB_COMMITMENTS_PER_BLOCK]
    partial_column: List[Cell, MAX_BLOB_COMMITMENTS_PER_BLOCK]
    kzg_proofs: List[KZGProof, MAX_BLOB_COMMITMENTS_PER_BLOCK]
    kzg_commitments: List[KZGCommitment MAX_BLOB_COMMITMENTS_PER_BLOCK]
    inclusion_proofs: List[Vector[Bytes32, SINGLE_KZG_COMMITMENT_INCLUSION_PROOF_DEPTH], MAX_BLOB_COMMITMENTS_PER_BLOCK]

```python
def verify_partial_data_column_sidecar(sidecar: PartialDataColumnSidecar) -> bool:
    """
    Verify if the KZG proofs are correct.
    """
    # The column index also represents the cell index
    cell_indices = [i for i, b in enumerate(cells_present_bitmap) if b]

    # Batch verify that the cells match the corresponding commitments and proofs
    verify_cell_kzg_proof_batch(
        commitments_bytes=sidecar.kzg_commitments,
        cell_indices=cell_indices,
        cells=sidecar.column,
        proofs_bytes=sidecar.kzg_proofs,
    )
    
def verify_data_column_sidecar_inclusion_proofs(sidecar: PartialDataColumnSidecar) -> bool:
    """
    Verify if the given KZG commitments included in the given beacon block.
    """
   return all(
      is_valid_merkle_branch(
           leaf=hash_tree_root(sidecar.kzg_commitments[i]),
           branch=sidecar.kzg_commitments_inclusion_proofs[i],
           depth=SINGLE_KZG_COMMITMENT_INCLUSION_PROOF_DEPTH,
           index=get_subtree_index(get_generalized_index(BeaconBlockBody, "blob_kzg_commitments", i)),
           root=sidecar.signed_block_header.message.body_root,
      )
      for i in range(len(sidecar.kzg_commitments))
   )

@fradamt, do you have any extra context on why gloas removes those fields useful for validating the column before the block in the DataColumnSidecar?

Yes, the thinking was that in gloas the beacon block is explicitly intended to come first, the columns are not even supposed to be sent before the beacon block has reached everyone (because the structure of the slot becomes propagate beacon block -> attest -> propagate payload and blobs -> ptc vote). However, for gloas purposes we could go back to "self-verifying columns" (kzg commitments + inclusion proof) if the added complexity turns out to not be worth it (I don't think the current work on gloas devnet0 has surfaced these issues, so maybe worth bringing it up to people working on it). Whether we do that or not doesn't change things for partial messages though, including all kzg commitments + inclusion proof is fine in a column since there's only one kzg commitment per cell, it is not fine in a small partial message.

[dknopik]

The block root is the partial message group_id, which is sent separately from the message data. So adding it to the message is redundant.