chore(testing): initial Kurtosis integration for exploratory migration #8296

IreneBa26 · 2025-08-29T16:05:57Z

Motivation

This PR explores an initial approach to migrating Lodestar's testing framework from Docker-based execution to Kurtosis-managed Ethereum network simulations. It proposes an abstraction layer that could preserve compatibility with existing test assertions while shifting the underlying infrastructure.
The core logic is maintaining the exact same test assertions while swapping the implementation from Docker containers to Kurtosis services.

This PR is opened for feedback and team discussion about the architectural approach before proceeding with the complete implementation.

This work is part of the Ethereum Protocol Fellowship - Cohort 6.
More details, including weekly updates on implementation and proposals, can be found in this documentation.

Description

Proposed Solution

Implement a Kurtosis SDK-based runner that provides:

Enclave Management: Isolated execution environments with automatic cleanup
Service Orchestration: Declarative network configuration via YAML
API Abstraction: Consistent interfaces regardless of underlying execution method
Backward Compatibility: Existing test assertions continue to work unchanged

Technical Implementation

1. Core Architecture (runner/)

KurtosisSDKRunner: Implements IRunner interface, inspired by Docker Runner pattern
Manages enclave lifecycle (create, start, stop, destroy)
Deploys services using ethpandaops/ethereum-package Starlark package
Maps Kurtosis services to NodeService abstraction layer
kurtosisTypes.ts: Type definitions for network configuration and service metadata
loadKurtosisConfig.ts: YAML configuration loader for test scenarios

2. Interface Layer (simulation/interfaces-kurtosis.ts)

Extended IRunner: Modified to accept KurtosisNetworkConfig instead of JobOptions[]
Service Mapping: KurtosisServicesMap provides typed access to Kurtosis-managed services
Backward Compatibility: Maintains existing NodePair, BeaconNode, ExecutionNode interfaces
Kurtosis-Specific Extensions: New interfaces like BeaconNodeKurtosis for future use

3. Simulation Integration (simulation/simulation-kurtosis.ts)

initWithKurtosisConfig(): Static factory method for Kurtosis-based simulations
Service Conversion: Maps Kurtosis NodeService objects to Crucible-compatible NodePair structures
API Client Integration: Creates Lodestar API clients using Kurtosis service endpoints
Lifecycle Management: Integrates with existing simulation tracker and epoch clock
createNodePairsFromKurtosis(): Converts Kurtosis services to Crucible NodePair[]
createBeaconNodeFromKurtosis(): Creates BeaconNode with Kurtosis service context

4. Test Infrastructure (test/)

multi-fork.yml: Example network configuration demonstrating multi-client setups
runner-test.ts: Test suite for KurtosisSDKRunner
Service creation and lifecycle validation
Port mapping and network topology verification
Service role inference and metadata extraction

Data Flow Architecture

YAML Config → KurtosisNetworkConfig → KurtosisSDKRunner → KurtosisServicesMap → NodeService[] → NodePair[] → Test Assertions

Example of a possible workflow:

flowchart TD
    A["multiFork.test.ts"] --> B["Simulation.initWithKurtosisConfig()"]
    B --> C["Simulation constructor"]
    C --> D["new Kurtosis Runner()"]
    
    D --> E["multiFork.yml"]
    E --> F{"Configuration Defines"}
    F --> G["EL/CL Types & Counts"]
    F --> H["Additional Services<br/>e.g., dora"]
    F --> I["Fork Epochs<br/>Altair, Bellatrix, Capella,<br/>Deneb, etc."]
    
    D --> K["Loads YAML"]
    K --> L["Launches Enclave"]
    L --> M{"Returns"}
    M --> N["Beacon API URLs"]
    M --> O["EL/CL RPC Endpoints"]
    M --> P["Service Identifiers<br/>cl-1, el-2, etc."]
    
    N --> Q["Simulation"]
    O --> Q
    P --> Q
    Q --> R["Passes to Assertions"]
    R --> S["No API Change Required"]

Future Work

If team feedback is positive on the architectural approach:

Investigate if and how Kurtosis services can be fully replicated into compatible NodePair objects
Research compatibility between Kurtosis service endpoints and existing Lodestar API interfaces (IRunner, BeaconNode, etc)
Attempt to fully replicate one existing simulation test using Kurtosis instead of Docker

Note: The current approach only explains the architectural strategy. Actual NodePair replication from Kurtosis services is not yet implemented and requires further investigation to determine feasibility and approach

Feedback Request

This PR is opened specifically for team feedback and discussion on:

Runner & Architecture: Is the current Kurtosis runner implementation and overall substitution strategy the right approach?
Interface & Types: Do the proposed Kurtosis types/abstractions fit well, or should they be rethought?
Migration Path: Does the step-by-step transition plan look correct, or should it be structured differently?

**Motivation** Exploratory work for replacing Docker-based simulation infra with Kurtosis. Goal is to validate feasibility and document integration points with the current test framework **Description** - Add KurtosisSDKRunner to wrap enclave lifecycle and expose services (working) - Add runner-test.ts to run the KurtosisSDKRunner and print service info from Kurtosis (working) - Add simulation-kurtosis.ts and interfaces-kurtosis.ts as mock scaffolding → contain errors / unused params, not usable, shared only for feedback - Add kurtosisTypes.ts and helper loadKurtosisConfig.ts for typing and YAML parsing **Notes** Only KurtosisSDKRunner and runner-test.ts are functional Other files are placeholders for structure review and not production-ready

CLAassistant · 2025-08-29T16:06:03Z

All committers have signed the CLA.

nazarhussain · 2025-09-02T08:00:18Z

packages/cli/test/utils/crucible/kurtosis/simulation/interfaces-kurtosis.ts

@@ -0,0 +1,497 @@
+import {ChildProcess} from "node:child_process";


Seems this file contains a lot of duplicate interfaces from other interface.ts file. Keep it consistent and add new interfaces in the interface.ts.

nazarhussain · 2025-09-02T08:05:18Z

packages/cli/test/utils/crucible/kurtosis/test/multi-fork.yml

@@ -0,0 +1,15 @@
+participants:


We also need to update relevant multiFork.ts file so instead of inline declerations of nodes, it loads up the Yaml file.

nazarhussain · 2025-09-02T08:05:48Z

packages/cli/test/utils/crucible/kurtosis/runner/kurtosisSDKRunner.ts

+   *
+   * enclaveName: Default name for the Kurtosis enclave - can be overridden
+   */
+  constructor(enclaveName = "crucible-enclave") {


Make the name required, so the enclave name already be unique with the simulation running. And don't mixup the services among different simulations.

nazarhussain · 2025-09-02T08:06:01Z

packages/cli/test/utils/crucible/kurtosis/runner/kurtosisSDKRunner.ts

+        id: serviceName,
+        serviceContext,
+        beaconApiUrl: ports.get("http") ? `http://localhost:${ports.get("http")?.number}` : undefined,
+        roles: this.inferRoles(serviceName),


Can there be a service with multiple roles? I think not.

nazarhussain · 2025-09-02T08:06:13Z

packages/cli/test/utils/crucible/kurtosis/runner/loadKurtosisConfig.ts

+  }
+
+  const raw = await fs.readFile(fullPath, "utf8");
+  return parse(raw) as KurtosisNetworkConfig;


Is there a way to validate the kurtosis config at this point? may be some helper function in their SDK.

The create() function calls runStarlarkRemotePackageBlocking(), which in turn invokes sanity_check() and input_parser() from the ethereum-package.

These already provide an initial validation of the Kurtosis config. Is this sufficient?

If sufficient but should occur earlier, one possible adjustment could be to move the validation (happening in simulation-kurtosis.ts) to run right after await env.runner.start(sim-${id}); and before const services = await env.runner.create(kurtosisConfig);, so that it happens before create() rather than inside it

nazarhussain · 2025-09-02T08:06:29Z

packages/cli/test/utils/crucible/kurtosis/simulation/interfaces-kurtosis.ts

+  Nethermind = "execution-nethermind",
+}
+
+export enum ExecutionStartMode {


We may delete this one now. Not needed ny more.

nazarhussain · 2025-09-02T08:06:51Z

packages/cli/test/utils/crucible/kurtosis/test/runner-test.ts

@@ -0,0 +1,258 @@
+/**


Remove this file from this PR and keep it locally for your testing.

- Update NodeService to use single role instead of multiple boolean flags - Change beaconApiUrl to generic apiUrl for better abstraction - Enclave name changed to be unique with the simulation running - Addressed kurtosis config validation - Added new interfaces in the interface.ts - Add TODO comment indicating future merge of simulation-kurtosis into simulation - Updated multiFork.test.ts wiith the new kurtosis configuration - Removed runner-test.ts from the PR - Improved multi-fork.yml parameters to be similar to the multifork.test.ts config

nazarhussain · 2025-09-18T18:28:31Z

.gitignore

+# Kurtosis test files (keep locally, don't commit)
+packages/cli/test/utils/crucible/kurtosis/test/runner-test.ts
+


We would remove this line when this PR get ready.

nazarhussain · 2025-09-18T18:29:22Z

packages/cli/test/utils/crucible/kurtosis/simulation/simulation-kurtosis.ts


  //❌ REMOVE - Docker-specific
-  async createNodePair<B extends BeaconClient, V extends ValidatorClient, E extends ExecutionClient>({
+  /*async createNodePair<B extends BeaconClient, V extends ValidatorClient, E extends ExecutionClient>({


You can remove the code which is not needed from this file anyway.

As previously suggested for interfaces-kurtosis.ts to keep it consistent and append new code, should simulation-kurtosis.ts be unified into simulation.ts and just rely on interfaces.ts + simulation.ts, removing simulation-kurtosis.ts and interface.ts. entirely?

packages/cli/test/utils/crucible/kurtosis/test/multi-fork.yml

- Update simulation-kurtosis.ts - Removed runner-test.ts from the PR - Removed Docker Runner elements - Aligned multi-fork.yml parameters to the multifork.test.ts config, as in unstable branch

nazarhussain reviewed Sep 2, 2025

View reviewed changes

nazarhussain requested changes Sep 2, 2025

View reviewed changes

nazarhussain reviewed Sep 18, 2025

View reviewed changes

packages/cli/test/utils/crucible/kurtosis/test/multi-fork.yml Outdated Show resolved Hide resolved

refactor(kurtosis): network config improvements

2ec1b92

- Update simulation-kurtosis.ts - Removed runner-test.ts from the PR - Removed Docker Runner elements - Aligned multi-fork.yml parameters to the multifork.test.ts config, as in unstable branch

		@@ -0,0 +1,497 @@
		import {ChildProcess} from "node:child_process";

		# Kurtosis test files (keep locally, don't commit)
		packages/cli/test/utils/crucible/kurtosis/test/runner-test.ts

Uh oh!

chore(testing): initial Kurtosis integration for exploratory migration #8296

Are you sure you want to change the base?

chore(testing): initial Kurtosis integration for exploratory migration #8296

Uh oh!

Conversation

IreneBa26 commented Aug 29, 2025

Motivation

Description

Proposed Solution

Technical Implementation

Data Flow Architecture

Future Work

Feedback Request

Uh oh!

CLAassistant commented Aug 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

CLAassistant commented Aug 29, 2025 •

edited

Loading