Skip to content

Conversation

@snowmead
Copy link
Contributor

@snowmead snowmead commented Dec 10, 2025

Added a new Dynamic Network integration test runner which can startup a network of an arbitrary number of nodes (i.e. any number of BSPs, MSPs, Fishermans and Users).

Each MSP and Fisherman will internally startup a Postgres and Indexer container to simulate actual production architecture.

Keys are generated JIT and deleted after tests complete.

A single integration test was created simulating an MSP and multi BSP network responding to a single storage request, essentially testing if the MSP can distribute the file to the many BSPs without fail.

test/util/netLaunch/dynamic/
├── index.ts              # Public exports
├── topology.ts           # NetworkTopology types + validation
├── dynamicLauncher.ts    # Main launcher + DynamicNetworkContext
├── serviceGenerator.ts   # Docker Compose service generation
├── portAllocator.ts      # Sequential port allocation
├── keyGenerator.ts       # Identity generation + key injection
├── connectionPool.ts     # Lazy API connections with LRU
└── progressReporter.ts   # Startup progress feedback

High level example:

import { describeNetwork } from "../util/bspNet/testrunner";

describeNetwork(
  "100 BSP scale test",
  { bsps: 100, msps: 2, fishermen: 1, users: 3 },
  { timeout: 600000 },
  (ctx) => {
    ctx.it("all BSPs can volunteer for storage", async () => {
      // Get the block producer (only BSP-0 can seal in dev mode)
      const producer = await ctx.network.getBlockProducerApi();

      // Map over all BSPs to verify connectivity
      const results = await ctx.network.mapBsps(async (api, index) => {
        const peerId = await api.rpc.system.localPeerId();
        return { index, peerId: peerId.toString() };
      });

      assert.equal(results.length, 100);
    });

    ctx.it("MSPs can accept storage requests", async () => {
      const mspApi = await ctx.network.getMspApi(0);
      const mspId = ctx.network.getMspProviderId(0);
      // ... test logic
    });
  }
);

@snowmead snowmead added B5-clientnoteworthy Changes should be mentioned client-related release notes D3-trivial👶 PR contains trivial changes that do not require an audit not-breaking Does not need to be mentioned in breaking changes labels Dec 10, 2025
@snowmead snowmead marked this pull request as draft December 11, 2025 19:42
@snowmead snowmead force-pushed the feat/dynamic-integration-test-network-topology branch 2 times, most recently from 3ec84a6 to 1dac00e Compare December 12, 2025 20:26
Add infrastructure for launching test networks with configurable topology:
- Arbitrary numbers of BSPs, MSPs, fishermen, and users
- Per-node configuration (RocksDB, capacity, custom args)
- Lazy connection pooling with LRU eviction
- Sequential port allocation to avoid conflicts
- Phased startup with silent progress reporting

Key components:
- describeNetwork() test runner for topology-based tests
- DynamicNetworkContext with typed API accessors
- Docker Compose service generation
- Cryptographic identity generation for nodes

Includes benchmark tests for 10-BSP scale and MSP file distribution.
@snowmead snowmead force-pushed the feat/dynamic-integration-test-network-topology branch from c1f5878 to d561788 Compare December 12, 2025 21:09
@snowmead snowmead marked this pull request as ready for review December 12, 2025 21:16
@santikaplan santikaplan requested a review from ffarall December 17, 2025 14:19
Resolved conflict in test/package.json by:
- Keeping cross-env prefix from main for cross-platform compatibility
- Preserving additional test scripts from feature branch (test:integration-benchmark, test:unit)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

B5-clientnoteworthy Changes should be mentioned client-related release notes D3-trivial👶 PR contains trivial changes that do not require an audit not-breaking Does not need to be mentioned in breaking changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants