Skip to content

fix(proxy)!: MaxBlobSizeBytes processing #1762

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jul 18, 2025

Conversation

ethenotethan
Copy link
Contributor

@ethenotethan ethenotethan commented Jul 13, 2025

Why are these changes needed?

  • Fixes debug log to present # of bytes post payload -> blob encoding
  • Reduce config ingestion for digital information unit to only support a maximum unit of mib with only base2

Checks

  • I've made sure the tests are passing. Note that there might be a few flaky tests, in that case, please comment that they are not relevant.
  • I've checked the new test coverage and the coverage percentage didn't drop.
  • Testing Strategy
    • Unit tests
    • Integration tests
    • This PR is not tested :(

@Copilot Copilot AI review requested due to automatic review settings July 13, 2025 21:14
@ethenotethan ethenotethan requested a review from a team as a code owner July 13, 2025 21:14
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR refines blob size validation and limits supported byte units in configuration parsing.

  • Refactors the max-blob-size check to use a named encodedBloblen variable and corrects the error log to report the post‐encoding length.
  • Removes support for gigabyte and terabyte units (gb, gib, tb, tib) in ParseBytesAmount, now only allowing KB, MB, MiB.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
api/proxy/store/generated_key/eigenda/eigenda.go Introduces encodedBloblen and updates the oversized-blob error message
api/proxy/common/common.go Deletes GB/TB unit cases in ParseBytesAmount, defaulting unsupported units
Comments suppressed due to low confidence (3)

api/proxy/common/common.go:67

  • Consider updating this error message to list the supported units (e.g., kb, mb, mib) and update any related docs or examples to reflect the removal of GB/TB units.
		return 0, fmt.Errorf("unsupported unit: %s", unit)

api/proxy/common/common.go:63

  • Add or update unit tests for ParseBytesAmount to cover the supported units (kb, mb, mib) and ensure it returns errors for removed units like gib or tb.
		return uint64(num * 1024 * 1024), nil

api/proxy/store/generated_key/eigenda/eigenda.go:116

  • [nitpick] Either remove or implement this TODO by relocating the blob length validation into PutBlob to improve cohesion and reduce duplication.
	// TODO: We should move this length check inside PutBlob

@ethenotethan ethenotethan changed the title fix(proxy): MaxBlobSizeBytes processing fix(proxy): MaxBlobSizeBytes processing Jul 13, 2025
litt3
litt3 previously approved these changes Jul 14, 2025
@@ -63,14 +63,6 @@ func ParseBytesAmount(s string) (uint64, error) {
return uint64(num * 1024 * 1024), nil
case "mb":
return uint64(num * 1000 * 1000), nil // Decimal megabyte
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should just remove the base 10 versions of these units... they just don't have a place in our system.

Practically, it would be too easy for someone to use MB and mean binary, but get decimal.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ethenotethan ethenotethan changed the title fix(proxy): MaxBlobSizeBytes processing !fix(proxy): MaxBlobSizeBytes processing Jul 15, 2025
@ethenotethan ethenotethan changed the title !fix(proxy): MaxBlobSizeBytes processing fix(proxy)!: MaxBlobSizeBytes processing Jul 15, 2025
@ethenotethan ethenotethan requested review from litt3 and samlaf July 17, 2025 19:20
@ethenotethan ethenotethan merged commit 86e27fa into master Jul 18, 2025
30 checks passed
@ethenotethan ethenotethan deleted the ethenotethan--fix-proxy-max-size-processing branch July 18, 2025 04:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants