feat(cosmos): Chunked bundle installation

[kriskowal]

Closes: #11634
Supersedes: #11202

Description

This change introducers machinery to agd such that chains can receive bundles in chunks. The Go side of the house stages these chunks until a number of blocks have elapsed, such that if all the chunks arrive before the deadline, the accumulated bundle will pass up to the SwingSet controller, just as it would if it were submitted in a single transaction. This allows us to receive large bundles despite immovable limitations on the maximum RPC message size.

The change also makes support for this feature evident by adding governable swingset parameters for the maximum bundle size and maximum chunk size, such that clients like Cosgov and sense support for chunked bundles and size them appropriately.

Alternative design considered

In this rendition, we use an incremental artifact ID counter selected by the chain. This approach limits our ability to do a dry run in the future, since the submitted artifacts have an ID that depends on the first transaction. We have elected to ship this and reserve the right to create another lane later that would support a dry run, in which case the submitter would use a consistent hash that closes over both the configured chunk size limit and the hashes of every individual chunk, then obligate the server to verify the submitted manifest and update the deadline if it is resubmitted before expiration.

Security Considerations

This change uses content hashes to ensure that the assembled bundles match the manifest of the original submission. Consequently, chunks can theoretically be submitted by multiple parties, but because they are addressed by hash, an attack would have to find a coïncident hash for a chunk of the same exact size.

Scaling Considerations

This will cause some data to buffer before its integrity is verified. This should impose limited overhead. Chunks are staged in a linked list to avoid expensive seeking while collecting expired chunks.

Documentation Considerations

The change will not be documented but accompanied with a change to Cosgov DCFoundation/cosmos-proposal-builder#70 that transparently uses the feature if present on the connected chain.

Testing Considerations

We have elected to manually test. To test this feature, run a local chain with this branch and also run the associated instance of Cosgov. This change will be removed from draft status when the Cosgov change is integrated.

Upgrade Considerations

Should not impact upgrade. However, publishing clients can sense whether this feature is present on the connected chain by looking for the SwingSet params for chunk size limits.

[socket-security]

No dependency changes detected. Learn more about Socket for GitHub.

👍 No dependency changes detected in pull request