chore: Update release 2.1 from master #1772
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* remove ext data type * fix: Oversized blob test * fix: Oversized blob test - use smaller blob * remove prover with verifier * remove prover with verifier * chore: Update G1 default value * fix: Resolve commitment decoding bug * Added badge for current actions. Signed-off-by: Cody Littley <[email protected]> * Split actions into smaller workflows. Signed-off-by: Cody Littley <[email protected]> * Remove omnibus actions workflow. Signed-off-by: Cody Littley <[email protected]> * Fixed holesky test name Signed-off-by: Cody Littley <[email protected]> * Updated readme with more badges. Signed-off-by: Cody Littley <[email protected]> * Rename some things. Signed-off-by: Cody Littley <[email protected]> * feat: Optimism v1.9 support * backup * Update README.md Co-authored-by: Ethen <[email protected]> * Update store/router.go Co-authored-by: Ethen <[email protected]> * PR review * read from eigenda first, failover to s3 * docs(README.md): fix typo * chore: lagniappe v1.4 updates * chore: lagniappe v1.4 updates - fix readme * chore: lagniappe v1.4 updates - update README.md * replace g1.point with larger g1.point that supports 32MiB (#87) * replace g1.point with larger g1.point that supports 32MiB * change flag related part to blob size --------- Co-authored-by: Ubuntu <[email protected]> * Fix syntax in op-devnet-allocs.sh The ending "fi" for the if-statement was accidentally omitted. * chore: go fmt * fix: rpc server metric label shortened to commitment header * chore: EigenDA V8 dep bump * chore: EigenDA V8 dep bump - update large blob test * chore: EigenDA V8 dep bump - bump eigenda to incorporate tls disperser read client fix && add linting * chore: EigenDA V8 dep bump - consolidate linting actions * chore: EigenDA V8 dep bump - bump lint version * chore: EigenDA V8 dep bump - update to latest versoin * chore: Update runners * chore: Update lint install * chore: Update lint install * chore: Update lint install * chore: rm gosec from lint cfg * Update lint.yml * chore: Update commitment schemas in README * chore: Update commitment schemas in README - update env ingestion * chore: less verbose PR template * chore: Update eigenda dep to v0.8.1 && update tests for 16MB blobs (#100) * chore: Update eigenda dep to v0.8.1 && update tests for 16MB blobs * chore: Update eigenda dep to v0.8.1 && update tests for 16MB blobs - fix lint * refactor(metrics): remove dependency on optimism metrics (#94) * refactor(metrics): remove dependency on optimism metrics Also add status code label to http_requests_total metric * doc: added TODO comment for updating commitmentHeader in metric middleware * fix: lint accomodate -> accommodate * fix: lint * docs: fix file misnaming (#103) * change disperseBlob response timeout to 60sec (#105) * change disperseBlob response timeout to 60sec * fix README --------- Co-authored-by: Ubuntu <[email protected]> * feat: Support explict modes for read fallbacks and caching (#104) * feat: Support explict modes for read fallback and caching * feat: Support explict modes for read fallback and caching - updated docs and fixed bugs * feat: Support explict modes for read fallback and caching - updated docs and fixed bugs * feat: Support explict modes for read fallback and caching - updated E2E test * feat: Support explict modes for read fallback and caching - updated E2E test * feat: Support explict modes for read fallback and caching - updated cfg validation tests * feat: Support explict modes for read fallback and caching - incorporate PR feedback * feat: Support explict modes for read fallback and caching - update query timeout * chore: Update large blob test to use 16mb (#108) * chore: Update large blob test to use 16mb * chore: Update large blob test to use 16mb - fix cache retrieval bug * chore: Update large blob test to use 16mb - update docs * fix: Commitment decoding bug && server handler tests (#112) * fix: Commitment decoding bug && svr handler tests * fix: Commitment decoding bug && svr handler tests - lint fix * fix: Commitment decoding bug && svr handler tests - mv gosec to golangci lint * fix: Commitment decoding bug && svr handler tests - rm false positives * Feat memstore artificial latency (#114) * chore: make server flags with default value optional * chore: fix run-server command (renamed to run-memstore-server) * feat: put/get artificial latency for memstore backend * doc: add memstore put/get latency flags into README options table * chore: clean flags * refactor: use config for memstore * fix: memstore tests * refactor: add default verifier config creation function for memstore tests * fix(lint): remove unused var * chore: rename memstore eventLoop -> pruningLoop (#117) * BREAKING suggestion: change dockerfile to use entrypoint instead of cmd (#116) breaking: change dockerfile to use entrypoint instead of cmd * Chore give docker image fullname (#119) * chore: docker-build uses full ghcr image name * chore: clean dockerignore file * feat: Redis backend target (#111) * feat: Redis backend target * feat: Redis backend target - update holesky test make target * feat: Redis backend target - address PR feedback * feat: Redis backend target - rebase to new main * Update Makefile: add disperse-test-blob (#115) * Update Makefile: add disperse-test-blob This'll make it easier to test that the da-proxy is working manually. eg: ``` make run-memstore-server make disperse-test-blob ``` * Update Makefile * chore(breaking): add EIGENDA_PROXY prefix to memstore flags (#121) * chore: add EIGENDA_PROXY prefix to memstore flags needed for consistency with all other flags * fix: add EIGENDA_PROXY prefix in the .env example files * fix: wrap crit entrypoint error for easier debugging (#122) * fix: negative confDepth bug in verifier (#131) * fix: negative confDepth bug in verifier * fix: lint * fix: request record label with commitment mode and version (#109) * fix: request record label with commitment mode and version * fix: lint * refactor: combine request metrics and write bad requests * feat: added docker compose services and provisioning * fix: commitment version type * refactor: commitment mode and version meta struct * chore: passing gosec with nosec directives * ci: add tests on expected commitment mode and cert version * fix: comments and labels * chore: bunch of small refactors for improved devex (#124) * fix: Dockerfile syntax warning (as -> AS) * fix: injected ldflags * chore: tag make build-docker image with :dev * chore: better error logging * fix: lint * refactor: single var block declaration * fix: server_tests * Fix s3 panic from keccak commitment put requests (#129) * refactor: e2e server_test to separate testsuite config creation from testsuite creation this will give flexibility to change the exact config being used to start the testsuite server * test: add e2e test for keccak commitment to expect error from s3 backend not set * fix: panic on nil ptr dereference when s3 backend not set * fix: lint * fix: e2e tests * dep: update eigenda to latest version for latest eigenda-client logging behavior (#132) * Fix metric middleware panic (#133) * test: add server_test middleware test * fix: revert to old put/get handler behavior, and fix tests * fix: metric panic * fix: lint * fix: HandlePut uses hardcoded default version (#135) * chore: remove unused tls logic (#139) * Eigenda client subsystem logger (#136) * format: middleware function signatures for better readability * chore: wrap an error for easier debugging * chore: give eigenda-client its own logger with subsystem label * initial set of edge cases (#110) * initial set of edge cases * commenting out test checking for nil * fixing for ci lint * ci fix * removing unwanted assert * more fixes to tests * removing commented code * separating tests for integration only * changing test name and adding OP issue link * Update e2e/server_test.go Co-authored-by: Samuel Laferriere <[email protected]> * changed test name and fn name * test config changes * making the linter happy * adding comments --------- Co-authored-by: anupsv <[email protected]> Co-authored-by: Samuel Laferriere <[email protected]> * fix: custom error wrapped around error and commitment meta (#134) * fix: custom error wrapped around error and commitment meta * refactor: minor improvement for custom errors * fix: lint * refactor: clean up useless handlerError * fix: Return 400 when blobs exceed max limit (#140) * Fix config checker logic (#138) * remove -v from make test * refactor: config checker one rule was broken due to new semantics of confDepth flag, which can't be negative anymore * fix: config tests * fix: lint * feat: add new eigenda-cert-verification-enabled flag * fix: config tests * docs: update readme with new cert-verification-enabled flag info * show response commitment (#144) Co-authored-by: Ubuntu <[email protected]> * rename commitment mode (#151) * rename * restore go.mod and go.sum --------- Co-authored-by: Ubuntu <[email protected]> * ci: give holesky-test workflow access to secrets via pull_request_target (#153) * Update Optimism dependency to OP-Stack 1.9.2 (#150) Updates: * optimism v1.9.0 -> v1.9.2 * op-geth to the commit 110c433a2469 which is used by optimism v1.9.2 * removed unnecessary version replacements * go-kzg-4844 now uses v1.0.0 * pebble now uses v1.1.2 Necessary code changes: * opio is now ctxinterrupt * plasma is now alt-da * refactor: clean up flags and configs (#146) * refactor: move flags from config.go -> flags.go * refactor: move flags+config to new cli package * refactor: stores into subdirs (precomputed_key and generated_key) * fix: memstore test * refactor: move config back to server package + rename cli->flags * refactor: move redis flags+config into redis package cli.go * refactor: move s3 flags to their own package cli.go * refactor: move eigenda_client flags to their own (temporary) package * refactor: move verifier flags/config into its own package cli.go * fix: lint Also added WaitForFinalization eigenda flag * refactor: move memstore flags to their own package cli.go * fix: e2e holesky-test * fix: e2e tests * fix: e2e tests (attempt 2) * refactor: rename KeyGeneratedStore -> GeneratedKeyStore * style(flags): remove unneeded explicit "" values * style: add category to all s3 flags (forgot in previous commit) * style: lint * feat: update eigenda latest release v0.8.4 (#158) * fix: verifier srs flags paths (#159) * fix: verifier srs flags paths * style: lint * fix: import and expose the verifier flags (#161) * fix: import and expose the verifier flags * fix: maxBlobLength flag action was not being run * style: print config on startup * fix: don't allow to configure fallback or cache with nil Redis|S3 ins… (#155) fix: don't allow to configure fallback or cache with nil Redis|S3 instances * Added --s3.enable-tls flag (#148) * Added --s3.enable-tls flag * Update S3 env vars * chore: Don't return commitment for OP Keccak256 mode on PUT requests (#147) * chore: Only return commitment on PUT for OP Generic and Simple commitment modes * chore: Only return commitment on PUT for OP Generic and Simple commitment modes - update tests * chore: Only return commitment on PUT for OP Generic and Simple commitment modes * chore: Only return commitment on PUT for OP Generic and Simple commitment modes- rm error file * chore: add deprecated flags to force early crash when used (#165) * chore: add deprecated flags to force early crash when used * flags: update memstore flags category description * flags: add verifier deprecated flags + fix some other small things * flags: add deprecation warnings to old memstore flag env-vars * lint: fix lint warning * flags: remove wait_for_finalization from deprecated flags didn't exist before, so shouldnt be deprecated - its a new flag * flags: cleaner deprecated warning (env var appears without surrounding []) * flags: change withEnvPrefix to return single string for better deprecated error msgs * fix: lint caught bug * opsec: prevent private key from being logged * update readme and template (#170) Co-authored-by: Ubuntu <[email protected]> * hide eth rpc, format hidden string (#171) Co-authored-by: Ubuntu <[email protected]> * docs: better usage docs for signer-private-key flag (#174) * Disable chunk encoding for `put` requests to Google Cloud Storage (#167) * Set DisableContentSha256=true for Put() requests to Google * Move putObjectOptions to s3 Store struct * fix env variable in docker compose (#180) Co-authored-by: Ubuntu <[email protected]> * Allow Google storage endpoints without http/s (#179) Fix Google URL check * fix: change put route put/ -> put (#168) * fix: change put route put/ -> put * chore: update dependencies optimism1.9.2->1.9.3 and go1.21->1.22 * fix: readd the op-geth replace directive * style: lint * fix: change client to use new put route instead of put/ * dependency: update op to commit that has put route fix * fix(routes): still need /put/ route * fix(ci): update go1.21->1.22 everywhere (#183) We updated go.mod to use go1.22 recently, but forgot to update the ci workflows. * Improve local test setup (#177) * tests: add vscode settings to run tests from vscode * tests: run minio and redis as testcontainers instead of starting from makefile * style: fix lint * test: pin minio version to RELEASE.2024-10-02T17-50-41Z * docs(main_test.go): more accurate comment * docs: add docstring above startRedisContainer function * style: startMinio -> startMinIO * docs: add comment with TODO for refactoring global init() starting test containers * refactor: cleaner routing using gorilla mux regexps (#185) * refactor: GET routes to use gorilla mux regexp patterns * refactor: server into files routing/handlers/middleware * style: fix lint * refactor: use gorilla mux for POST routes * routing: fix incorrect error msgs for simple commitments * cleanup: old unused function * tests: fix e2e tests * docs: add docstrings to handlers * style: use vars for the gorilla mux routing variables instead of hardcoded strings * logging: better logging msgs in op handlers * metrics: use "unknown" instead of "noCommitment" in metrics middleware when missing meta info * logging(handlers): simplify by moving "processing request" logs to shared handlers only * fix: fetch quorum parameters from edasm at RBN instead of latest block (#190) * fix: fetch quorum parameters from edasm at RBN instead of latest block * docs: fix getQuorumAdversaryThreshold comment * feat: Optional async decoupling for secondary writes and reworked E2E metric assertions (#182) * chore: Better abstract secondary storage * chore: Better abstract secondary storage - add channel stream for secondary insertions * chore: Better abstract secondary storage - add channel stream for secondary insertions * chore: Better abstract secondary storage - observe secondary storage via metrics * chore: Better abstract secondary storage - observe secondary storage via metrics - cleanups * chore: Better abstract secondary storage - observe secondary storage via metrics - refactors and lints * chore: Better abstract secondary storage - observe secondary storage via metrics - refactors and lints * chore: Better abstract secondary storage - observe secondary storage via metrics - refactors and lints * chore: Better abstract secondary storage - observe secondary storage via metrics - ensure thread safety for secondary stores * chore: Better abstract secondary storage - observe secondary storage via metrics - use in memory metrics * chore: Better abstract secondary storage - observe secondary storage via metrics - add concurrency flag * chore: Better abstract secondary storage - observe secondary storage via metrics - fmt * chore: Better abstract secondary storage - address PR feedback, add benchmarks, code comments * chore: Better abstract secondary storage - refactor tests * chore: Better abstract secondary storage - more test clean ups * Merge branch 'main' of github.com:Layr-Labs/op-plasma-eigenda into epociask--chore-reabstract-router * chore: Better abstract secondary storage - update go mock ref * chore: Better abstract secondary storage - address PR feedback * refactor: use batch confirmation that was upstreamed to eigenda client (#192) * refactor: verifier confirmation logic to use upstreamed blob confirmation depth guarantee flags: add new eigenda-client flags for blob confirmation depth also pass those flags to verifier (until we also upstream verification to the eda client) comment: was pointing to wrong eigenda-client repo in TODO comment fix: go.mod to point to PR commit instead of using local replace directive chore: go mod tidy to generate go.sum chore: use proto Getter functions instead of fields (that are potentially nil) ci: upgrade golangci-lint version 1.60->1.61 fix: verifySecurityParams func arguments after rebase chore: make more robust verifyBatchConfirmedOnchain logic Added retry logic and better comments style: Onchain -> OnChain docs: better comment describing eth_getBlockByNumber call args style: better error msg when memstore enabled but cert verification is not fix: verifier.WaitForFinalization was not set fix(flags): deleted deprecated flags that had same name as new ones in other package, causing panic style(flags): merged WaitForFinalizationFlagName into ConfirmationDepth flag It now accepts uint conf depth or 'finalized' string now chore: remove unused utils.EqualBytes function (same as stdlib exp function anyways) chore: remove log line added for debugging * fix: missing contexts in a few places (after rebase) * fix: lint issues * deps: update eigenda to commit 5fe3e910a22d after merging upstream PR * docs(verifier): expand explanation for reorg edge case * docs(verifier): make more precise explanation for why we need to retry checking batch confirmation depth * style: fix lint * fix(e2e tests): missing new eigenda-client required config fields (#196) * fix(e2e tests): missing new eigenda-client required config fields - ethrpc and svcmanageraddr * Revert "ci: give holesky-test workflow access to secrets via pull_request_target (#153)" This reverts commit 15b10fd0feaa324ae49c143cb642d1b3e39ffc26. The commit was doing things very wrong. I hadn't understood how pull_request_target works. Was causing the workflow to run against main branch head commit instead of PR commit. We will need to find another solution to the problem of letting external contributors run this workflow. * ide: update vscode settings.json with env vars to run holesky testnet e2e tests * docs: shorten svcManagerAddr holesky testnet comment * tests: add panic when both INTEGRATION and TESTNET env vars are set This forces test runner to be fully aware of which test suite he is running (otherwise it implicitly runs the TESTNET suite) * style: make handleGetShared returned error print hex encoded commitment Previously it was printing as byte array, which is unreadable and clutters logs * docs: better comments in metrics middleware * deps: upgrade eigenda dep to regression fix commit TODO: will need to update this after that PR is merged * deps: update eigenda dependency to master head Just merged https://github.com/Layr-Labs/eigenda/pull/849 which will fix our ci bug * fix(#191): Routing namespace --> Storage (#195) * feat(failover): return 503 to batcher when eigenda is down (#193) * feat(failover): return 503 to batcher when eigenda is down chore: go mod tidy to generate go.mod feat: dealing with new eigenda-client grpc errors + ErrorFailover convention comment: fix typo feat(handlers): postShared returns 429 when disperser rate limited client flag(eigenda): rename RetriesBeforeFailover -> PutRetries reviewer correctly pointed out that retrying was more general than only for failovers lint: nolint exhaustive switch check for Put case * flag(eigenda-client): add cli flag for new config ConfirmationTimeout * tests(handlers): rename servers_test.go -> handlers_test.go + some small refactors * tests(handlers): add PUT failure tests for all modes * test(handlers): remove unneeded expectedError in TestHandlerPut * dep: update eigenda to master head (contains ErrorFailover fix) * tests(handlers): add tests for error types (including failover) * fix: errors after rebase * flags: clearer usage string for eigenda-client ResponseTimeoutFlag * style: define is503 function to follow isABC pattern * style: make lint * fix: Hide other sensitive cfg values (#194) * fix: Hide other sensitive cfg values * fix: prettyPrintConfig hide values before marshalling * docs: add comment for why we are hiding password when marshalling * fix(makefile): run-memstore-server command missing new mandatory flags * fix: prettyPrintConfig was hiding wrong field. Change RPC->EthRpcUrl * refactor: way redis/s3 hides config details (use custom marshalling function) --------- Co-authored-by: Samuel Laferriere <[email protected]> * refactor: rename client -> SimpleCommitmentClient and add comments (#201) * refactor: rename client -> SimpleCommitmentClient and add comments * style: rename client/client.go -> client/simple.go * fix(middleware): proper handling of handler status codes in log/metrics middlewares (#200) * fix(middleware): proper handling of handler status codes in log/metrics middlewares * style: remove trailing whitespace (fix lint) * fix(middleware) cert version byte handling * style: make format * chore(simple_client): Add explicit ServiceUnavailable error type (#207) * chore(simple_client): Add explicit ServiceUnavailable error type * chore(simple_client): Add explicit ServiceUnavailable error type - bring back constructor * chore(simple_client): Add explicit ServiceUnavailable error type - fix lint * chore: update op dependency to v1.9.5 (#203) * chore: update op dependency to v1.10.0 * chore: downgrade to v1.9.5 + fix optimism tests * docs: update README with failover info (#204) * docs: update README with failover info * docs(README): add link to arb failover spec * docs(README): add "Service Unavailable" description for 503 status code * fix: Disable `SERVICE_MANAGER_ADDR` & `ETH_RPC` as mandatory (#205) * fix: Disable svc manager address & eth rpc as mandatory * fix: Disable svc manager address & eth rpc as mandatory - add eigenda client override when memstore enabled * chore(simple_client): Add explicit ServiceUnavailable error type - address PR comment and add unit test * chore(simple_client): Add explicit ServiceUnavailable error type - address PR comment * chore: Add mockable http client to simple client (#209) * feat: separate simple client into its own module (#208) * feat: separate simple client into its own module This will make it easier to import it into other projects without importing all of the other dependencies of our root go.mod. We will first merge this PR with a replace directive in the main go.mod, then release v0.1.0 of the new client module, and then make a new PR to use that version in the main go.mod file. * chore: SimpleClient -> Standard Client (#210) --------- Co-authored-by: Ethen <[email protected]> * fix: copy `client/go.mod` to `builder` in Dockerfile (#211) fix: copy client/go.mod to builder * fix(CI): Add docker build and curl check (#214) * Create dependabot.yml (#216) * Create dependabot.yml * Update dependabot.yml * Update dependabot.yml * Update .github/dependabot.yml * Add fuzz tests (#137) * Fix single node tree verification bug (#219) * Fix single node tree bug * Clarify method doc Signed-off-by: litt3 <[email protected]> * Add link to on chain method Signed-off-by: litt3 <[email protected]> --------- Signed-off-by: litt3 <[email protected]> * fix: memstore returns data_length in number of symbols instead of bytes (#223) * make memstore return data_lenght in number of symbols * fix lint * lint * docs: fix readme test docs, and add test comments to makefile (#225) * docs: fix readme test documentation, and add comments describing tests to makefile Signed-off-by: litt3 <[email protected]> * Add missed period Signed-off-by: litt3 <[email protected]> * Add additional explanation of e2e tests Signed-off-by: litt3 <[email protected]> --------- Signed-off-by: litt3 <[email protected]> * fix(sigmap-EDAP-04): Add IO parsing size constraint on incoming request bodies (#227) * fix(sigmap-EDAP-04): Add IO parsing size constraint on incoming request bodies * fix(sigmap-EDAP-04): Add IO parsing size constraint on incoming request bodies - address PR feedback * fix(sigmap-EDAP-04): Miscellaneous General Comments (#228) * fix(sigmap-EDAP-04): Miscellaneous General Comments * fix(sigmap-EDAP-04): Miscellaneous General Comments - address PR feedback * fix(sigmap-EDAP-01): Missing nil Checks On Parameters Of Incoming Requests (#231) * fix(sigmap-EDAP-01): Missing nil Checks On Parameters Of Incoming Requests * fix(sigmap-EDAP-01): Missing nil Checks On Parameters Of Incoming Requests - check all nil fields and use proto accessors for referencing cert fields * fix(sigmap-EDAP-06): Missing IsOnCurve & IsInSubgroup Checks For Elli… (#229) * fix(sigmap-EDAP-06): Missing IsOnCurve & IsInSubgroup Checks For Elliptic Curve Point * fix(sigmap-EDAP-06): Missing IsOnCurve & IsInSubgroup Checks For Elliptic Curve Point - remove unnecessary function * fix(sigmap-EDAP-06): Missing IsOnCurve & IsInSubgroup Checks For Elliptic Curve Point - address PR feedback * fix(sigmap-EDAP-06): Missing IsOnCurve & IsInSubgroup Checks For Elliptic Curve Point - address PR feedback * chore: Add security audit (#234) * fix: Field element casting to affine representation (#240) * adding audit report (#244) * adding audit report * Create SECURITY.md * removing from root * fixing repo name --------- Co-authored-by: anupsv <[email protected]> * docs: update readme shields to point to actual workflows (#248) * fix(#251): better error logging for RPC lookup errors against service manager (#254) * chore: modify verifier to not require eth archive node (#241) * chore: force verifier's EthConfirmationDepth to be <64 We panic in the flag's action, as well as in the verifier's constructor when this condition is not respected. This will make sure that an archival node is not required. * chore: modify verifier to load quorum parameters only once at initialization This removes the need for running with an eth archive node. * style: fix minor lint issue * docs: update README to mention that archival node is no longer needed * docs: clean-up README archival node requirement explanation * docs: fix verify/cert.go comment typo Co-authored-by: Ethen <[email protected]> * docs: for eg -> e.g. * style(cert): remove unecessary bound checks from inside loop * style: create consts package with EthHappyPathFinalizationDepthBlocks = 64 * style: change panic into error return * docs: change op reference for eth reference * docs: make flag comment simpler * Update verify/cert.go Co-authored-by: EthenNotEthan <[email protected]> --------- Co-authored-by: Ethen <[email protected]> * docs: pimp out readme with blob lifecycle diagrams (#233) * chore: move pull_request_template.md under .github/ dir * docs: reorder README sections to feel more natural (move flags to bottom) * docs (wip): add blob lifecycle diagrams to README * docs: remove Sidecar from README title (proxy is not necessarily a side) * docs: add blob lifecycle section to README * docs: add TOC to README * style(routing): rename raw_commitment -> payload We changed the name in README so should be consistent in the code * docs: update README sections to use Payload instead of Blob Posting Blobs -> Posting Payloads Retrieving Blobs -> Retrieving Payloads * docs: remove "obviously" word * Required quorums glitch (#255) * Avoid quorum 1 check on range of Holesky blocks * Improve SVC address check * Update verify/verifier.go Co-authored-by: Samuel Laferriere <[email protected]> * Update verify/verifier.go Co-authored-by: Samuel Laferriere <[email protected]> * Avoid unnecessary cast * Rename constant * Fix lint --------- Co-authored-by: Samuel Laferriere <[email protected]> * docs: update README posting payload image (#256) * fix: remove last eth_call that required archive node (#259) Forgot this one in https://github.com/Layr-Labs/eigenda-proxy/pull/241 * docs: update SECURITY.md with audit commit + fix small typos (#263) * docs: update SECURITY.md * docs: update SECURITY.md with audited release number + release where findings were addressed * chore: Update compose with secondary store setup && minor refactors (#270) * chore: Update compose with secondary store setup && minor refactors * chore: Minor code comment * chore: Update logging to use `layr-labs/eigensdk-go` (#264) * chore: Update logging to use eigengo-sdk * chore: Update logging to use eigengo-sdk - fmt * chore: Update logging to use eigengo-sdk - incorporate latest eigenda changes, deprecate flags, and update docs / code comments * chore: Update logging to use eigengo-sdk - fix test * chore: Update logging to use eigengo-sdk - svc_binding -> edsm_binding * chore: Update logging to use eigengo-sdk - move logging to own packag e and copy entire dependency package vs hyrbid * chore: Update logging to use eigengo-sdk - rm todo in metrics && fix lint errs * chore: Update logging to use eigengo-sdk - refactor flag construction patterns across subpackages to adhere to uniform abstraction * chore: Update logging to use eigengo-sdk - refactor flag construction patterns across subpackages to adhere to uniform abstraction * chore: Update logging to use eigengo-sdk - update metric * chore: Update logging to use eigengo-sdk - go gfmt * chore: Update logging to use eigengo-sdk - apppease white space linting * chore: Update logging to use eigengo-sdk - white space lint * chore: Update logging to use eigengo-sdk - white space lint * docs: add srs requirements section to readme (#276) * docs: add srs requirements section to readme * docs: fix readme typo * feat: public s3 config option (#277) * Create codeql-scanning.yml (#279) * Create codeql-scanning.yml * adding versions * chore: fix miscellaneous typos in comments (#287) * ci: add test to make sure env vars in .env.example.holesky are valid (#286) * ci: add test to make sure env vars in .env.example.holesky are valid * ci: add a ping to proxy's health endpoint in test-proxy-startup-with-env-vars script * chore: bump default confirmation-depth 0->8 blocks (#285) * chore: change default confirmation-depth to 5 blocks (from 0 blocks) This is a very important change. Reorgs of 1 block are quite common and can cause a da cert to be submitted to the rollup inbox with a wrong confirmationBlock! It's important to wait a few blocks for the confirmBatch to have landed onchain before submitting the dacert to the batcher inbox. * docs: add troubleshooting guide with first section on the batch-hash-mismatch error * chore: bump default confirmationDepth 5->8 * feat: add memstore config to return failover errors on puts (#289) * chore: add comments to release-drafter.yml files to explain what they are * chore: add memstore config to return failover errors on puts This is useful for testing failover in batcher. Right now a new proxy would need to be spun up with this config turned on. We should consider adding an admin rpc rpi to turn these on/off dynamically. * chore: remove unecessary "(FOR TESTING") prefix in some memstore flags * ci: cleanup into a single per-pr.yml gha * Revert "ci: cleanup into a single per-pr.yml gha" This reverts commit adfdfb80b3dcec0c55e7e13ffc27f758fa0a5385. * ci: add trailofbits/go-queries to codeql-scanning.yml (#288) * Update codeql-scanning.yml * Update codeql-scanning.yml * feat: add http routes and handlers to get/patch the memstore config (#293) * feat: add http routes and handlers to get/patch the memstore config * test(memstore_handlers): fix GET tests * style: make format * style(memconfig): remove unneeded tags from struct in MarshalJSON * chore: add marshalJSON function to memstore safeconfig (it wasnt getting logged on startup) * style(verifier): better startup logs * chore: fix golangci.yml errors/warnings Not sure why the ci error started appearing just now... * chore(makefile): clean up irrelevant run-memstore-server flags * test: fix fuzz test (CreateTestSuite was not registering the route after the refactor) * Update store/generated_key/memstore/README.md Co-authored-by: ethenotethan <[email protected]> * docs(memstore): add command to print memstore flags --------- Co-authored-by: ethenotethan <[email protected]> * ci: cleanup into a single per-pr.yml gha (#295) * ci: cleanup into a single per-pr.yml gha * chore: fix golangci.yml issues * ci: rename gchr -> push-ghcr.yml * ci: rename per-pr workflow to CI (readme badge looks better) * docs: update README with new ci badges * feat: Memconfig client (#301) * feat: Memconfig client * feat: Memconfig client - demarcate clients into seperate packages && rename module to * feat: Memconfig client - update Dockerfile to use new clients package instead of /client * feat: Memconfig client - update docs * feat: Memconfig client - update Dockerfile * feat: Memconfig client - update Dockerfile * feat: Memconfig client - rm client dependency from go.mod * feat: Memconfig client - update Dockerfile/dockerignore * feat: Memconfig client - add usability comment * chore(memstore): hash entire cert + remove cert verification (#308) * chore(memstore): use hash of cert for key * chore(memstore): use hash of cert for key * chore(memstore): Compute batch header hash before persisting (#311) * chore(memstore): Compute batch header hash before persisting * chore(memstore): Compute batch header hash before persisting - lint * chore(memstore): Compute batch header hash before persisting - use encoded value * chore: fix readme with correct cast cmd (#314) Co-authored-by: Samuel Laferriere <[email protected]> * fix: E2E holesky tests (#299) * fix: E2E holesky tests * fix: E2E holesky tests - lint && update oversize checks * fix: E2E holesky tests - reduce holesky times --------- Co-authored-by: Samuel Laferriere <[email protected]> * test: fix failing e2e-local safety check (#323) e2e tests were turned off by mistake for a while. https://github.com/Layr-Labs/eigenda-proxy/pull/299 just turned them back on. Not sure why the test was only caught by CI after merging. But this commit fixes the failing test. * ci: separate holesky-test into its own job (#325) * ci: separate holesky-test into its own test Also lower its timeout to 30mins Holesky is broken so this test is flaky. Makes sense to sepraate it into its own job so that we can rerun only this test when it fails. * ci: remove 30 min timeout on workflow in favor of that in makefile wasnt seeing logs because ci job was getting cancelled before test failed, and didn't have -v flag enabled. So enabled the flag and removed the timeout from ci in case * feat!: add eigenda v2 support (#321) * feat: Initial V2 scaffolds * feat: Initial V2 scaffolds - add boilerplate dependency injection and feature flag * feat: Initial V2 scaffolds - go fmt * feat: Initial V2 scaffolds - wire up disperser and retriever clients * feat: EigenDAV2 commitment processing and generation (#265) * fix(#251): better error logging for RPC lookup errors against service manager (#254) * chore: modify verifier to not require eth archive node (#241) * chore: force verifier's EthConfirmationDepth to be <64 We panic in the flag's action, as well as in the verifier's constructor when this condition is not respected. This will make sure that an archival node is not required. * chore: modify verifier to load quorum parameters only once at initialization This removes the need for running with an eth archive node. * style: fix minor lint issue * docs: update README to mention that archival node is no longer needed * docs: clean-up README archival node requirement explanation * docs: fix verify/cert.go comment typo Co-authored-by: Ethen <[email protected]> * docs: for eg -> e.g. * style(cert): remove unecessary bound checks from inside loop * style: create consts package with EthHappyPathFinalizationDepthBlocks = 64 * style: change panic into error return * docs: change op reference for eth reference * docs: make flag comment simpler * Update verify/cert.go Co-authored-by: EthenNotEthan <[email protected]> --------- Co-authored-by: Ethen <[email protected]> * docs: pimp out readme with blob lifecycle diagrams (#233) * chore: move pull_request_template.md under .github/ dir * docs: reorder README sections to feel more natural (move flags to bottom) * docs (wip): add blob lifecycle diagrams to README * docs: remove Sidecar from README title (proxy is not necessarily a side) * docs: add blob lifecycle section to README * docs: add TOC to README * style(routing): rename raw_commitment -> payload We changed the name in README so should be consistent in the code * docs: update README sections to use Payload instead of Blob Posting Blobs -> Posting Payloads Retrieving Blobs -> Retrieving Payloads * docs: remove "obviously" word * Required quorums glitch (#255) * Avoid quorum 1 check on range of Holesky blocks * Improve SVC address check * Update verify/verifier.go Co-authored-by: Samuel Laferriere <[email protected]> * Update verify/verifier.go Co-authored-by: Samuel Laferriere <[email protected]> * Avoid unnecessary cast * Rename constant * Fix lint --------- Co-authored-by: Samuel Laferriere <[email protected]> * docs: update README posting payload image (#256) * fix: remove last eth_call that required archive node (#259) Forgot this one in https://github.com/Layr-Labs/eigenda-proxy/pull/241 * docs: update SECURITY.md with audit commit + fix small typos (#263) * docs: update SECURITY.md * docs: update SECURITY.md with audited release number + release where findings were addressed * feat: EigenDAV2 commitment processing and generation * feat: EigenDAV2 commitment processing and generation - add note describing follow up todo --------- Co-authored-by: Samuel Laferriere <[email protected]> Co-authored-by: Gaston Ponti <[email protected]> * feat: Initial V2 scaffolds - dependency injection working, current bug with G2 point ingestion * feat: Initial V2 scaffolds - E2E proof of concept sort've * feat: Initial V2 scaffolds - bump EigenDA * feat: Initial V2 scaffolds - update dependency injection * feat: Initial V2 scaffolds - hardcode quorums * feat: Initial V2 scaffolds - working mvp * feat: Initial V2 scaffolds - working lint and tests * feat: Initial V2 scaffolds - add todo around G2 ingestion * feat: Initial V2 scaffolds - change all usage of directory path to just path * feat: Initial V2 scaffolds - refactor manager * feat: Initial V2 scaffolds - refactor commitments * chore(V2): Refactor dependency injection && config mgmt (#296) * chore(V2): Refactor dependency injection && config mgmt * chore(V2): Refactor dependency injection && config mgmt - address config.go feedback * chore(V2): Refactor dependency injection && config mgmt - update polynomial form flag * chore(V2): Refactor dependency injection && config mgmt - loader --> builder * feat: Initial V2 scaffolds - add verification && more verbose logging * feat: Initial V2 scaffolds - update e2e setup to use memconfig * chore(V2): update config checks and fix custom quorum injection (#298) * feat: Initial V2 scaffolds - update e2e setup to use memconfig * chore(V2): update config checks and fix custom quorum injection * chore(V2): update config checks and fix custom quorum injection - update flag descriptions && add flag * chore(V2): update config checks and fix custom quorum injectiom - address PR feedback * chore(V2): update config checks and fix custom quorum injectiom - update tls flag description * chore(V2): update config checks and fix custom quorum injectiom - pc --> proxyCfg * chore(V2): update config checks and fix custom quorum injectiom - private builder fields and comment * feat(v2): memstore (#305) * feat(V2): memstore * feat(V2): memstore - code comments, fix v2 wiring bugs, update monorepo dep * feat(V2): memstore - E2E tests * feat(V2): memstore - minor updates * feat(V2): memstore - address PR feedback * feat(V2): memstore - add CLI comment * feat(V2): memstore -address pr comments * feat: Initial V2 scaffolds - lint fixes and minor refactors * Update EigenDA version, and fix linter issues (#315) Signed-off-by: litt3 <[email protected]> * V2: fix e2e tests (#316) fix(E2E): Update oversized cert test to assert lookup error instead * Rework v2 initialization (#317) * Rework v2 initialization Signed-off-by: litt3 <[email protected]> * Use snake case for imports Signed-off-by: litt3 <[email protected]> * Adopt latest eigenDA changes Signed-off-by: litt3 <[email protected]> * Fix error messages Signed-off-by: litt3 <[email protected]> * Tweak import Signed-off-by: litt3 <[email protected]> * Check signer account balance Signed-off-by: litt3 <[email protected]> --------- Signed-off-by: litt3 <[email protected]> * Rename EigenDACommit to EigenDACommitmentType Signed-off-by: litt3 <[email protected]> * Strive for naming consistency Signed-off-by: litt3 <[email protected]> * Do general cleanup Signed-off-by: litt3 <[email protected]> * Fix bug found by copilot Signed-off-by: litt3 <[email protected]> * Further separate out secrets Signed-off-by: litt3 <[email protected]> * Reduce diff size Signed-off-by: litt3 <[email protected]> * Return error instead of panicking Signed-off-by: litt3 <[email protected]> * Fix log Signed-off-by: litt3 <[email protected]> * Only read v2 config if enabled Signed-off-by: litt3 <[email protected]> * Put back check for redis password Signed-off-by: litt3 <[email protected]> * Make configs not use pointers Signed-off-by: litt3 <[email protected]> * Remove reference to arbitrum Signed-off-by: litt3 <[email protected]> * Clean up Builder Signed-off-by: litt3 <[email protected]> * Do more cleanup on StorageManagerBuilder Signed-off-by: litt3 <[email protected]> * Move CertVerifierAddress into config struct Signed-off-by: litt3 <[email protected]> * Return struct instead of interface Signed-off-by: litt3 <[email protected]> * Convert len to symbols Signed-off-by: litt3 <[email protected]> * Add TODO Signed-off-by: litt3 <[email protected]> * Describe future improvements Signed-off-by: litt3 <[email protected]> * Add additional config check Signed-off-by: litt3 <[email protected]> --------- Signed-off-by: litt3 <[email protected]> Co-authored-by: Ethen Pociask <[email protected]> Co-authored-by: Samuel Laferriere <[email protected]> Co-authored-by: Gaston Ponti <[email protected]> * chore: better errors and docs for batch-hash-mismatch error (#322) * chore: create structured error for batch-hash-mismatch error This permits handling it. Realized while implementing this that there is currently no good way to deal with the error however... but decided to still commit it for better documentation of why this is not possible. * docs(troubleshooting_v1.md): expand on batch-hash-mismatch error documentation Explain that this is only possible on op-stack. * docs(troubleshooting_v1): add explanation that batch-hash-mismatch also affects nitro l3s * docs(batch-hash-mismatch): remove TODO since this error is not solvable Added explanation for why not. * docs(batch-hash-mismatch): update old incorrect documentation for BatchMetadataHashMismatchError Not actually possible to handle this error. Update the explanation, and point to new protobuf docs * docs: fix typo * fix: move errors.go to verify/v1 (it was left alone and breaking build after merge master) * docs: add readme link to v2 integration spec (#329) * chore: add linter to enforce max line lengths of 100 (#331) * chore: add golines to format line lengths to 100 This is enforced by the lll linter that we add to the golangci-lint config * chore: change golines formatter to use 120 max line len * style: make format * chore(golangci): allow certain patterns to have long lines in linter config * chore: break up all long lines or add `//nolint: lll` directive * docs: cleanup readme by separating users from contributors (#330) * docs: cleanup readme by separating users from contributors first time contributors were lost and didn't understand how to setup proxy and interact with it. Added a user guide to the readme which gives a quick start example of spinning up proxy in memstore mode, dispersing and retrieving a payload. Also refactored a bunch of the sections and TOC to make the whole readme easier to parse. * docs(README): rephrase GET route documentation * docs(README): break up long lines * docs: add godoc testable examples for the standard client (#336) Also added some godocs for eigenda-proxy itself and some other packages. * chore: Update to latest core version, to include cert verifier address provider (#333) Update to latest core version Signed-off-by: litt3 <[email protected]> * fix: Enforce V1 cert field sizes (#338) * Do integration test cleanup (#337) * Do some test cleanup Signed-off-by: litt3 <[email protected]> * Tweak more configurations Signed-off-by: litt3 <[email protected]> * Tweak log levels Signed-off-by: litt3 <[email protected]> * Add makefile doc Signed-off-by: litt3 <[email protected]> * Use better flag name Signed-off-by: litt3 <[email protected]> * Fix test type Signed-off-by: litt3 <[email protected]> * Add test matrix Signed-off-by: litt3 <[email protected]> * Tweak per pr yaml Signed-off-by: litt3 <[email protected]> * Remove irrelevant vars from settings.json Signed-off-by: litt3 <[email protected]> * Fix fuzz description Signed-off-by: litt3 <[email protected]> * Remove coverage from CI Signed-off-by: litt3 <[email protected]> * Fix test timeout Signed-off-by: litt3 <[email protected]> * Don't bother excluding fuzz tests from unit tests Signed-off-by: litt3 <[email protected]> * Decrease boilerplate Signed-off-by: litt3 <[email protected]> * Rename Environment to Backend Signed-off-by: litt3 <[email protected]> * Remove test matrix util Signed-off-by: litt3 <[email protected]> * Split benchmark test Signed-off-by: litt3 <[email protected]> * Try alternate test config strategy Signed-off-by: litt3 <[email protected]> * Fix failing tests Signed-off-by: litt3 <[email protected]> * test(fuzz): fix fuzz tests (#339) Cleaned up the fuzz tests to actually do something Reduced seed corpus to 3 meaningful inputs Removed cluttering log outputs so we can actually see the fuzzer output as its finding new interesting cases * Modify comment Signed-off-by: litt3 <[email protected]> * Use constant Signed-off-by: litt3 <[email protected]> --------- Signed-off-by: litt3 <[email protected]> Co-authored-by: Samuel Laferriere <[email protected]> * fix: Standard output check (#346) * fix: Standard output check * fix: Standard output check - make * fix: Standard output check - ignore version * fix: Standard output check - update sed cmd for ubuntu compatibility * fix: Standard output check - update sed cmd for ubuntu compatibility * Config only (#344) * Do some test cleanup Signed-off-by: litt3 <[email protected]> * Tweak more configurations Signed-off-by: litt3 <[email protected]> * Tweak log levels Signed-off-by: litt3 <[email protected]> * Add makefile doc Signed-off-by: litt3 <[email protected]> * Use better flag name Signed-off-by: litt3 <[email protected]> * Fix test type Signed-off-by: litt3 <[email protected]> * Add test matrix Signed-off-by: litt3 <[email protected]> * Tweak per pr yaml Signed-off-by: litt3 <[email protected]> * Remove irrelevant vars from settings.json Signed-off-by: litt3 <[email protected]> * Fix fuzz description Signed-off-by: litt3 <[email protected]> * Remove coverage from CI Signed-off-by: litt3 <[email protected]> * Fix test timeout Signed-off-by: litt3 <[email protected]> * Don't bother excluding fuzz tests from unit tests Signed-off-by: litt3 <[email protected]> * Decrease boilerplate Signed-off-by: litt3 <[email protected]> * Rename Environment to Backend Signed-off-by: litt3 <[email protected]> * Remove test matrix util Signed-off-by: litt3 <[email protected]> * Split benchmark test Signed-off-by: litt3 <[email protected]> * Try alternate test config strategy Signed-off-by: litt3 <[email protected]> * Fix failing tests Signed-off-by: litt3 <[email protected]> * test(fuzz): fix fuzz tests (#339) Cleaned up the fuzz tests to actually do something Reduced seed corpus to 3 meaningful inputs Removed cluttering log outputs so we can actually see the fuzzer output as its finding new interesting cases * Draft config cleanup Signed-off-by: litt3 <[email protected]> * Clean up Signed-off-by: litt3 <[email protected]> * Configure secrets correctly Signed-off-by: litt3 <[email protected]> * Iterate Signed-off-by: litt3 <[email protected]> * Iterate again Signed-off-by: litt3 <[email protected]> * Get relay registry address correctly Signed-off-by: litt3 <[email protected]> * Share code pathways between tests and production Signed-off-by: litt3 <[email protected]> * Clean up Signed-off-by: litt3 <[email protected]> * Fix docker Signed-off-by: litt3 <[email protected]> * Redact additional sensitive fields Signed-off-by: litt3 <[email protected]> * Disable verification for memstore mode Signed-off-by: litt3 <[email protected]> * Further simplify test configuration Signed-off-by: litt3 <[email protected]> * Fix test path Signed-off-by: litt3 <[email protected]> * Reenable test Signed-off-by: litt3 <[email protected]> * Fix more test paths Signed-off-by: litt3 <[email protected]> * Fix config tests Signed-off-by: litt3 <[email protected]> * Increase holesky e2e timeout Signed-off-by: litt3 <[email protected]> * Support preprod tests Signed-off-by: litt3 <[email protected]> * Fix yml Signed-off-by: litt3 <[email protected]> * Clean up Signed-off-by: litt3 <[email protected]> * Properly configure g2 Signed-off-by: litt3 <[email protected]> * Checkout with lfs Signed-off-by: litt3 <[email protected]> * Try to get SRS Order right Signed-off-by: litt3 <[email protected]> * Revert SRS changes Signed-off-by: litt3 <[email protected]> * Temporarily disable dispersal verification Signed-off-by: litt3 <[email protected]> * Try to fix fuzz test Signed-off-by: litt3 <[email protected]> * Revert unnecessary changes Signed-off-by: litt3 <[email protected]> * Use experimental g2 point loading Signed-off-by: litt3 <[email protected]> * Revert change to G1 Signed-off-by: litt3 <[email protected]> * Remove power of 2 g2 points Signed-off-by: litt3 <[email protected]> * Add deprecated flag Signed-off-by: litt3 <[email protected]> * Track deprecated flags Signed-off-by: litt3 <[email protected]> * Remove unnecessary edge case Signed-off-by: litt3 <[email protected]> * Fix broken V2 verify Signed-off-by: litt3 <[email protected]> * Reenable client side proving Signed-off-by: litt3 <[email protected]> * Declutter make tests Signed-off-by: litt3 <[email protected]> * Fix the test filter regex Signed-off-by: litt3 <[email protected]> * Update core version Signed-off-by: litt3 <[email protected]> * Try again to get the correct make command Signed-off-by: litt3 <[email protected]> * Prevent grep from buffering Signed-off-by: litt3 <[email protected]> * Use pipefail Signed-off-by: litt3 <[email protected]> * Give up on filtering Signed-off-by: litt3 <[email protected]> * Fix error message Signed-off-by: litt3 <[email protected]> * Add clarifying comment to v2 config struct Signed-off-by: litt3 <[email protected]> * Rename to MetricsConfig Signed-off-by: litt3 <[email protected]> * Fix terrible misnaming Signed-off-by: litt3 <[email protected]> * Simplify steps for overriding test flag configs Signed-off-by: litt3 <[email protected]> * Remove unused member var Signed-off-by: litt3 <[email protected]> * Fix small typos Signed-off-by: litt3 <[email protected]> * Make backend error message better Signed-off-by: litt3 <[email protected]> * Improve explanation of kzg behavior Signed-off-by: litt3 <[email protected]> * Improve v2 terminology and consistency Signed-off-by: litt3 <[email protected]> * Improve config ordering Signed-off-by: litt3 <[email protected]> * Revert name change Signed-off-by: litt3 <[email protected]> * Improve const names Signed-off-by: litt3 <[email protected]> * Change flag deprecation strategy Signed-off-by: litt3 <[email protected]> * First pass Signed-off-by: litt3 <[email protected]> * Move utility method Signed-off-by: litt3 <[email protected]> * Revert formatting changes Signed-off-by: litt3 <[email protected]> * Add Enabled to memstore config marshal Signed-off-by: litt3 <[email protected]> * Revert benchmark changes Signed-off-by: litt3 <[email protected]> * More reversions Signed-off-by: litt3 <[email protected]> * Move method in Setup Signed-off-by: litt3 <[email protected]> * Revert flags changes Signed-off-by: litt3 <[email protected]> * Newline change Signed-off-by: litt3 <[email protected]> * Revert formatting changes Signed-off-by: litt3 <[email protected]> * Revert secondary source changes Signed-off-by: litt3 <[email protected]> * Revert newline Signed-off-by: litt3 <[email protected]> * Fix tests Signed-off-by: litt3 <[email protected]> * Fix kzg config init Signed-off-by: litt3 <[email protected]> * Revert grep addition Signed-off-by: litt3 <[email protected]> * Fix test Signed-off-by: litt3 <[email protected]> * Remove accidental grep Signed-off-by: litt3 <[email protected]> * Try to get flags right again Signed-off-by: litt3 <[email protected]> * Split out test suite Signed-off-by: litt3 <[email protected]> * Split out verifier and kzg flags Signed-off-by: litt3 <[email protected]> * Shuffle around enable flag Signed-off-by: litt3 <[email protected]> * Update help_out Signed-off-by: litt3 <[email protected]> * Remove version Signed-off-by: litt3 <[email protected]> * Add missed config value Signed-off-by: litt3 <[email protected]> * Fix lint Signed-off-by: litt3 <[email protected]> * Update flag categories Signed-off-by: litt3 <[email protected]> * Lengthen e2e local timeout Signed-off-by: litt3 <[email protected]> --------- Signed-off-by: litt3 <[email protected]> Co-authored-by: Samuel Laferriere <[email protected]> * chore(clients): retract v1.0.1 (#350) We published a `clients/v1.0.1` tag by mistake, which got picked up by pkg.go.dev, and now the latest version of the docs (pointed to in our README) leads to https://pkg.go.dev/github.com/Layr-Labs/eigenda-proxy/[email protected]/standard_client. This directive retracts that version, such that the docs (and go commands) no longer point to this version (at this point v0.2.0 is the latest). Once we reach semver v1.0.1, we can possibly remove this retract directive (??), or just skip this number and go to v1.0.2. * docs(readme): document 1account/proxy (#353) * docs(readme): document 1account/proxy * docs(readme): clarify single key per rollup statement * docs(readme): add repo+release info (#352) * Fix e2e tests (#351) * Fix tests Signed-off-by: litt3 <[email protected]> * Apply fix to fallback Signed-off-by: litt3 <[email protected]> * Try increasing timeout Signed-off-by: litt3 <[email protected]> * Increase test parallelism Signed-off-by: litt3 <[email protected]> * Increase timeout even more, hopefully enough Signed-off-by: litt3 <[email protected]> --------- Signed-off-by: litt3 <[email protected]> * Create migration tests (#348) * Create migration tests Signed-off-by: litt3 <[email protected]> * Fix unit tests Signed-off-by: litt3 <[email protected]> * Clean up Signed-off-by: litt3 <[email protected]> * Fix naming Signed-off-by: litt3 <[email protected]> * More naming fixes Signed-off-by: litt3 <[email protected]> * Hopefully final naming fixes Signed-off-by: litt3 <[email protected]> * More clean up Signed-off-by: litt3 <[email protected]> * Update core commit Signed-off-by: litt3 <[email protected]> * Move require out of utility method Signed-off-by: litt3 <[email protected]> * Add better comments describing switching dispersal Signed-off-by: litt3 <[email protected]> * Move where atomic version is constructed Signed-off-by: litt3 <[email protected]> * Modify backend init strategy Signed-off-by: litt3 <[email protected]> * Add additional require Signed-off-by: litt3 <[email protected]> * Implement new config enum for backends Signed-off-by: litt3 <[email protected]> * Fix unit test Signed-off-by: litt3 <[email protected]> * Update help_out Signed-off-by: litt3 <[email protected]> * Improve test assertions Signed-off-by: litt3 <[email protected]> * Add test util Signed-off-by: litt3 <[email protected]> * Reorganize v2 enablement flags Signed-off-by: litt3 <[email protected]> * Update help_out Signed-off-by: litt3 <[email protected]> * Check backends len Signed-off-by: litt3 <[email protected]> * Try increasing memstore e2e timeout. Unsure why it's not completing Signed-off-by: litt3 <[email protected]> * Revert "Try increasing memstore e2e timeout. Unsure why it's not completing" This reverts commit 5a72f6ff6bfac18f2e9a61943b5f45affc07089b. * Make it possible to enable v2 backend with v1 backend Signed-off-by: litt3 <[email protected]> * Use variadic param Signed-off-by: litt3 <[email protected]> --------- Signed-off-by: litt3 <[email protected]> * Create env var test (#349) * Create env var test Signed-off-by: litt3 <[email protected]> * Update core commit Signed-off-by: litt3 <[email protected]> * Fix script Signed-off-by: litt3 <[email protected]> * Try bogus signing key Signed-off-by: litt3 <[email protected]> * Increase wait time Signed-off-by: litt3 <[email protected]> * Increase wait time even more Signed-off-by: litt3 <[email protected]> * Increase wait time to 75 Signed-off-by: litt3 <[email protected]> * Try matrix strategy Signed-off-by: litt3 <[email protected]> * Fix test name Signed-off-by: litt3 <[email protected]> * Add check to make sure env path is specified Signed-off-by: litt3 <[email protected]> * Create separate env var tests for v1, v2, and both Signed-off-by: litt3 <[email protected]> * Update .env file in .envrc Signed-off-by: litt3 <[email protected]> --------- Signed-off-by: litt3 <[email protected]> * chore(make): rename build command (#359) * chore(make): rename build command Used to be "eigenda-proxy". Renamed for consistency with `docker-build` command. Also most people are used to just typing `make build`, which doesn't require knowing the binary name. * chore(make): fix gen-static-help-output was still pointing to prev `eigenda-proxy` target instead of new `build` target * Improve CI speed and clarity (#358) * Get preprod CI passing Signed-off-by: litt3 <[email protected]> * Speed things up Signed-off-by: litt3 <[email protected]> * Use got…
* chore: rename periphery to integrations * Update contracts/README.md Co-authored-by: ethenotethan <[email protected]> * fix: edit README --------- Co-authored-by: ethenotethan <[email protected]> Co-authored-by: Samuel Laferriere <[email protected]>
* Draft CLAUDE.md Signed-off-by: litt3 <[email protected]> * Decrease LOC that consitute 'large changes' Signed-off-by: litt3 <[email protected]> * Add optimizations to list of things to note in docs Signed-off-by: litt3 <[email protected]> * Denote contract bindings as autogen files to not be modified Signed-off-by: litt3 <[email protected]> * Import linter yaml automatically Signed-off-by: litt3 <[email protected]> * Add link to anthropic docs to explain @ annotation Signed-off-by: litt3 <[email protected]> * Remove ambiguous word 'touch' Signed-off-by: litt3 <[email protected]> * Fix file imports Signed-off-by: litt3 <[email protected]> * Remove command import for now Signed-off-by: litt3 <[email protected]> * Add go.mod to imports Signed-off-by: litt3 <[email protected]> * Simplify wrong working dir pitfall Signed-off-by: litt3 <[email protected]> * Remove fuzzy style rules, and add testing section Signed-off-by: litt3 <[email protected]> * Specify that humans write docs Signed-off-by: litt3 <[email protected]> * Clarify how bindings can be updated Signed-off-by: litt3 <[email protected]> * Mention TODO comments Signed-off-by: litt3 <[email protected]> * Import available project slash commands Signed-off-by: litt3 <[email protected]> * Address the 'yes man' Signed-off-by: litt3 <[email protected]> * Augment prime directives Signed-off-by: litt3 <[email protected]> * Fix line len Signed-off-by: litt3 <[email protected]> * Don't auto parse docs Signed-off-by: litt3 <[email protected]> --------- Signed-off-by: litt3 <[email protected]>
* feat: add generic AccountLedger interface
- Create AccountLedger interface for payment state management
- Add LocalAccountLedger implementation for client-side tracking
- Refactor Accountant to use unified AccountLedger interface
* refactor(Ac
countLedger): revert Accountant changes, rewrite AccountLedger interface and LocalAccountLedger
impl
* refactor(AccountLedger): rewrite unit tests and comments
* refactor(AccountLedger): rm LocalAccountLedger struct and impl
* Update icicle encoder go version and fix build issue * Update disperser/cmd/encoder/icicle.Dockerfile Co-authored-by: Samuel Laferriere <[email protected]> --------- Co-authored-by: Samuel Laferriere <[email protected]>
* feat: quorum specific metrics for metered bytes * fix: unit test mock calls
* feat: integrate address directory to eth reader - Add NewReaderWithAddressDirectory() for address directory-based initialization - ContractNames constants matching AddressDirectoryConstants.sol - optional AddressDirectoryFlag, BlsOperatorStateRetrieverFlag, and EigenDAServiceManagerFlag - validation to require either address directory OR both legacy flags - backward compatibility for existing deployments without address directory * feat: fix BLS_OPERATOR_STATE_RETRIEVER typo * refactor: config valid check, interface binding, getAddress0 * fix: check zero addr, rm unused binding, err msg * feat: address directory reader * refactor: abstract the getter * rebase: master; add check of contract names * refactor: remove old new eth reader/writer method * refactor: update flags and configs for address dir * fix: chuner config lint * fix: more lints * fix: more lints * fix: inabox config * refactor(ethClient): comment cleanup * refactor(EigenDADirectory): update names and validation * refactor(EigenDADirectory): always get latest list of contract names * fix(EigenDADirectory): add service manager flag back to retreiver * refactor(proxy): add address directory config * refactor(proxy): add address directory config in test setup * refactor(EigenDADirectory): rename more address directory to eigenda directory * refactor(EigenDADirectory): more renaming * fix(EigenDADirectory): relay inabox config * fix(EigenDADirectory): add back flags and configs * fix(EigenDADirectory): add back flags and configs * fix(EigenDADirectory): inabox config * fix(EigenDADirectory): make all three flags optional * chore(proxy): gen-static-help-output * refactor(EigenDADirectory): help msgs
* fix: structify address config * refactor: use list of keys * fix: fmt
* refactor: remove keys which can be derived, fix some naming, add a removal of key function for eigenda directory * fix: remove usage auth registry * fix: uncommited file
Remove duplicated resources files. Files were recently moved.
* Add utility for locking directories in LittDB. * made suggested changes
* fix: disperser_client v2 nil ptr bug legacyReply.Reservation can be nil, which causes a panic when it is dereferenced. I can reproduce this by running TestDispersalWithInvalidSignature() in correctness_test.go. * Fix correctness tests. * more fixes * revert unintentional changes * made suggested changes * Made suggested changes. Signed-off-by: Cody Littley <[email protected]> --------- Signed-off-by: Cody Littley <[email protected]> Co-authored-by: Samuel Laferriere <[email protected]>
* chore: gitignore * feat: unify payment processing with DebitSlip abstraction * refactor(payment): isolate debit slip changes * fix: lint * refactor: debitSlip remove ReceivedAt and add error types * fix: lint * refactor: move debit slip into new payment package
* fix: period record circular wrapping refresh * chore: add unit tests for period records getRelativePeriodRecord
* fix: disperser_client v2 nil ptr bug legacyReply.Reservation can be nil, which causes a panic when it is dereferenced. I can reproduce this by running TestDispersalWithInvalidSignature() in correctness_test.go. * Fix correctness tests. * more fixes * revert unintentional changes * Prepare for running as action. * Updated keyvar config * Fix workflow name * Add temporary trigger. * tweaks Signed-off-by: Cody Littley <[email protected]> * integrate slack bot * Intentional failure. * better notification * Don't skip Signed-off-by: Cody Littley <[email protected]> * Update secret env vars * Update timeout. * Remove push trigger. * made suggested changes * Made suggested changes. Signed-off-by: Cody Littley <[email protected]> * Made suggested changes * Remove debug code --------- Signed-off-by: Cody Littley <[email protected]> Co-authored-by: Samuel Laferriere <[email protected]>
* Initialize subgraph * Initialize subgraph * Initialize subgraph * Save payment subgraph state * rename ActiveReservation to CurrentReservation * delete unused file and update ubuntu version * fix * Use spec version 1.2.0 * Add query examples + more test coverage * Remove docker compose * Add scripts for updating abi and adds a CI check when subgraph tests run * Change CurrentReservation entity to Reservation * Remove ABIs * Add back ABIs
* Tweak how traffic generator is configured. * Try simple ticker * bugfix * debug log Signed-off-by: Cody Littley <[email protected]> * fix floating point math Signed-off-by: Cody Littley <[email protected]> * Fix variable ticker * Made suggested change --------- Signed-off-by: Cody Littley <[email protected]>
Disable preprod live tests due to DNS break
* fix: defensive NewCertBuilder Wasn't sanitizing its passed inputs before, which caused a very hard to debug error when deploying to preprod where the RegistryCoordinatorAddr was not populated on the load generator. * chore: load_generator makefile improvements Only rebuild when necessary, and make build the default command, as opposed to clean * fix: loadgen metrics nil ptr bug load generator was crashing when metrics were set to false. Believe this is a mistake in prometheus' promauto API. created an issue for this upstream (see comment in code), and added a hacky fix in our repo in the meantime. * test: NewTestClient uses RegistryCoordinatorAddr from ethReader Instead of using one that was read from the config, which was bug prone (and did lead to a bug) * chore: CertBuilder info logs eth-call arguments This is needed to debug this call. We had one failing on preprod which was failing with a "execution reverted". We should probably add eth-calls like this before every eth-call site. Unsure if we want info or debug yet... * test: remove RegistryCoordinator addr from TestClientConfig No longer needed as we read it from the ServiceManager. This is less bug prone. * chore: make fmt * fix: relative paths in test/v2/config files Before relative paths wrt test files were used in the config file (for SRSPaths), which only worked for the tests, but not for running the load-generator. I made the paths relative to the config file itself, so now they work for both tests and load-generators. I think it also makes understanding the config files easier (otherwise there's some mysterious relative path which doesn't make sense without knowing that the paths are meant to be interpreted wrt where the tests are located). * style(cert_builder): only log eth-call params on error This way we only log when actually needed (to debug), and don't clutter our happy-path logs for no reason. * chore: fix test/v2/Makefile test command
* Add permissions for claude action Signed-off-by: litt3 <[email protected]> * Let claude write to PRs and issues Signed-off-by: litt3 <[email protected]> * Limit runaway usage Signed-off-by: litt3 <[email protected]> --------- Signed-off-by: litt3 <[email protected]>
* Create eigenda-releaser.yaml * Update eigenda-releaser.yaml
Update eigenda-releaser.yaml
Update eigenda-releaser.yaml
Update eigenda-releaser.yaml
* Update eigenda-releaser.yaml * Update eigenda-releaser.yaml
Update eigenda-releaser.yaml
* Update eigenda-releaser.yaml * Update eigenda-releaser.yaml adding more checks * Update eigenda-releaser.yaml branch checks.
* feat: rework deployment scripts * fix: config issues * feat: update config, add gitkeep * feat: address comments * fix: name, delete some old scripts * feat: add docs * fix: ci
* refactor: make pointsIO DRY * refactor: pointsIO into a single generic function Also added documentation for most functions, and exposed new functions to read from uncompressed points files. * test: add pointsIO_test file Includes a few tests/benchmarks to test uncompressed files. All set to be skipped. The benchmarks weren't conclusive enough to warrant a change to uncompressed points file imo, though we should at least debate it a little bit. * chore: fix lint * docs: add TODO prefix to comment per review feedback * docs: fix parameter documentation for functions * refactor: rename start/end to startPoint/endPoint for clarity * refactor: rename readDesiredBytes to readBytes * docs: add inline comments for loop variables and logic Address review comments from litt3 on PR #1726: - Document that i is the workerIndex - Document that j is the pointIndex - Explain that the final worker reads the rest of the points * refactor: use io.SeekStart instead of magic number 0 Address review comment from bxue-l2 on PR #1726 * refactor: rename loop variables for clarity instead of using comments - Rename i to workerIndex - Rename j to pointIndex - Remove inline comments as the variable names are now self-documenting This addresses the same review feedback from litt3 on PR #1726 but uses better variable names instead of comments * refactor: extract goroutine function into named deserializePointsInRange Extract the anonymous goroutine function into a named function deserializePointsInRange that better describes its purpose of deserializing point data from bytes. This addresses the review comment from litt3 on PR #1726 about moving the function definition out of readPointSection. * docs(pointsIO): remove wrong comment
…#1718) * add blob-derivation-and-preimage-in-integration-spec * Apply suggestions from code review Co-authored-by: Samuel Laferriere <[email protected]> * update spec and restructure the hokulea section * claude read 6-secure-intg * claude proof read 2-op-hokulea-secure-integration * add link to glossary * update glossary * minor tweek * Apply suggestions from code review commit suggestion Co-authored-by: Samuel Laferriere <[email protected]> * improve spec doc --------- Co-authored-by: Samuel Laferriere <[email protected]> Co-authored-by: Samuel Laferriere <[email protected]>
* Revert "fix: Update disperser client to take in logger vs initializing default one for ntp clock (#1651)" This reverts commit 7f93aea. * Revert "refactor: ntp init logs but not panic (#1568)" This reverts commit 00256fd. * Revert "feat: validator/disperser clock synchrony (#1509)" This reverts commit 5c19ace. * fix: NewDispserserClient call Remove no longer needed log first argument * chore: go mod tidy
* Revert "refactor: account ledger interface incorporates debit slip (#1732)" This reverts commit b4cf3d9. * Revert "fix: period record circular wrapping refresh (#1731)" This reverts commit 1091f46. * Revert "fix: disperser_client v2 nil ptr bug (#1710)" This reverts commit a5acd93. * Revert "fix: disperser_client v2 should use default onDemandQuorums (#1700)" This reverts commit 005e0a6. * Revert "refactor: unify accountant debit slip (#1695)" This reverts commit a9c0868. * Revert "feat: AccountantLedger interface for next step unifications (#1694)" This reverts commit 354e42f. * Revert "feat: extract generic payment logic (#1693)" This reverts commit 1003b98. * Revert "fix: make disperser client backwards compatible (#1686)" This reverts commit 26c612f. * Revert "fix: payment state api doesn't fail at zero valued reservations/ondemand (#1682)" This reverts commit 2031c1d. * Revert "refactor: quorum specific metrics for metered bytes (#1668)" This reverts commit 7ac688d. * Revert "feat: GetPaymentStateForAllQuorums api impl (#1664)" This reverts commit ed89588. * Revert "refactor: core meterer consolidation (#1663)" This reverts commit 319a865. * Revert "refactor: onchain state interface (#1662)" This reverts commit d795dfa. * Revert "refactor: core meterer period record module (#1661)" This reverts commit afbef90. * Revert "refactor: payment common functions (#1653)" This reverts commit 36a3d33. * Revert "fix: period index calculation (#1641)" This reverts commit c229748. * Revert "refactor: use generic column name (#1626)" This reverts commit 4d1994e. * Revert "feat: payment onchain state interface (#1625)" This reverts commit 61e46ac. * Revert "feat: offchain quorum period record getter (#1620)" This reverts commit a309139. * Revert "feat: protobuf payment state api quorum specific variant (#1613)" This reverts commit 58acdfd. * Revert "fix: nil assignment to new account (#1612)" This reverts commit bcbf9f5. * Revert "feat: offchain batch writes (#1580)" This reverts commit ce4d2f5. * Revert "feat: validator authenticate on-demand request by disperser key (#1539)" This reverts commit f545b16. * fix: issues after series of reverts (scary) * chore: add back ErrZeroSymbols error This error was likely removed by mistake during a revert conflict fix from one of the previous set of reverted PRs. * style: fix lint * test: fix accountant_test hardcoded error string * Revert "Revert "fix: period index calculation (#1641)"" This reverts commit b8767ae.
* Fix thread pool leak. * Fix tests * test fixes
|
The latest Buf updates on your PR. Results from workflow Buf Proto / buf (pull_request).
|
dmanc
changed the title
samlaf
approved these changes
Jul 17, 2025
* chore: move test/integration_test under test/v1 * test: move eigenda_client_e2e_test to live test (makefile + ci) Also changed the rpc-url to use same as that from the v2 live tests because the 1rpc one was super flaky and our integration test ci was constantly needing to be restarting after 10 mins because of that. * test: add error to localstack container error everywhere
Fix proto getters Signed-off-by: litt3 <[email protected]>
Fix v2 thread pool leak.
ci: turn off failing proxy preprod tests preprod relay and operators no longer expose a public DNS, to minimize egress costs. We need to run these tests from inside the cluster, either using a self-hosted runner or k8s cron job live test.
* fix: node shouldnt require eigenda directory Preprod Validator was seeing 2025/07/16 22:21:34 application failed: failed to create address directory reader: address directory must be a valid hex address: When using the old config. EigenDADirectory was added, but should be backward compatible and just seamlessly fallback to using the provided EigenDAServiceManager passed in as argument when Directory is not present. We will later deprecate and remove the other flags but for now we want the flags to be backwards compatible. * style: fix docs to point to eigencloud * chore: log directory and servicemanager in warn log * style(node): error out on eigendadirectory misconfiguration
* Write on cache miss flag (#422) * Write on cache miss flag * Fix category * Update help * Update docs/help_out.txt Co-authored-by: Samuel Laferriere <[email protected]> * Update flag doc * Add e2e test for WriteOnCacheMiss flag * Fix lint --------- Co-authored-by: Samuel Laferriere <[email protected]> * style: fix comments in proxy golangci.yml --------- Co-authored-by: Gaston Ponti <[email protected]>
pakim249CAL
approved these changes
Jul 17, 2025
* feat: Add network enum * feat: Add network enum && read system contracts from EigenDA directory contract - gen static output * chore: address pr feedback
samlaf
approved these changes
Jul 17, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why are these changes needed?
Checks