feat(fisherman): 🌎 batch file deletions #485
snowmeadnetwork.yml
on: pull_request
setup
8s
Push node Docker image to Docker Hub
58s
Network Test: Zombienet (Parachain + Relay)
3m 11s
Network Test: Zombienet CLI
2m 10s
Integration Tests: Fisherman
13m 50s
Check Rust/TS bindings are up to date
1m 19s
Matrix: Integration Tests: BSPNet
Matrix: Node Tests: Solo Dev Node
Matrix: Integration Tests: FullNet
Matrix: Integration Tests: User
Push backend Docker image to Docker Hub
0s
Integration Tests: Solochain EVM
5m 42s
Matrix: Integration Tests: Backend
Annotations
17 errors, 1 warning, and 18 notices
|
Integration Tests: Solochain EVM
Process completed with exit code 1.
|
|
Should fetch payment streams using the SDK's MspClient:
test/test/suites/integration/solochain-evm/sdk-precompiles.test.ts#L481
AssertionError [ERR_ASSERTION]: SDK did not return a payment stream for the expected MSP
at TestContext.<anonymous> (/home/runner/_work/storage-hub/storage-hub/test/suites/integration/solochain-evm/sdk-precompiles.test.ts:481:7)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async Test.run (node:internal/test_runner/test:1054:7)
at async Suite.processPendingSubtests (node:internal/test_runner/test:744:7) {
generatedMessage: false,
code: 'ERR_ASSERTION',
actual: undefined,
expected: true,
operator: '=='
}
|
|
Should upload the file to the MSP through the backend using the SDK's StorageHubClient:
test/test/suites/integration/solochain-evm/sdk-precompiles.test.ts#L458
Error [HttpError]: HTTP 500 for GET http://127.0.0.1:8080/buckets/0xb1bf58ff2c2b53bdaddc9279116e0af3bdd77b7a656aaf65a0446242c769e1a0/files
at H.request (/home/runner/_work/storage-hub/storage-hub/sdk/core/dist/index.node.js:1:242583)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async TestContext.<anonymous> (/home/runner/_work/storage-hub/storage-hub/test/suites/integration/solochain-evm/sdk-precompiles.test.ts:458:24)
at async Test.run (node:internal/test_runner/test:1054:7)
at async Suite.processPendingSubtests (node:internal/test_runner/test:744:7) {
status: 500,
body: { error: 'Database error: Record not found' }
}
|
|
Should create a new bucket using the SDK's StorageHubClient:
test/test/suites/integration/solochain-evm/sdk-precompiles.test.ts#L316
AssertionError [ERR_ASSERTION]: MSP listBuckets should include the created bucket
at TestContext.<anonymous> (/home/runner/_work/storage-hub/storage-hub/test/suites/integration/solochain-evm/sdk-precompiles.test.ts:316:7)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async Test.run (node:internal/test_runner/test:1054:7)
at async Suite.processPendingSubtests (node:internal/test_runner/test:744:7) {
generatedMessage: false,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: '=='
}
|
|
Should fetch payment streams using the SDK's MspClient:
test/test/suites/integration/solochain-evm/sdk-precompiles.test.ts#L481
AssertionError [ERR_ASSERTION]: SDK did not return a payment stream for the expected MSP
at TestContext.<anonymous> (/home/runner/_work/storage-hub/storage-hub/test/suites/integration/solochain-evm/sdk-precompiles.test.ts:481:7)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async Test.run (node:internal/test_runner/test:1054:7)
at async Suite.processPendingSubtests (node:internal/test_runner/test:744:7) {
generatedMessage: false,
code: 'ERR_ASSERTION',
actual: undefined,
expected: true,
operator: '=='
}
|
|
Should upload the file to the MSP through the backend using the SDK's StorageHubClient:
test/test/suites/integration/solochain-evm/sdk-precompiles.test.ts#L458
Error [HttpError]: HTTP 500 for GET http://127.0.0.1:8080/buckets/0xb1bf58ff2c2b53bdaddc9279116e0af3bdd77b7a656aaf65a0446242c769e1a0/files
at H.request (/home/runner/_work/storage-hub/storage-hub/sdk/core/dist/index.node.js:1:242583)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async TestContext.<anonymous> (/home/runner/_work/storage-hub/storage-hub/test/suites/integration/solochain-evm/sdk-precompiles.test.ts:458:24)
at async Test.run (node:internal/test_runner/test:1054:7)
at async Suite.processPendingSubtests (node:internal/test_runner/test:744:7) {
status: 500,
body: { error: 'Database error: Record not found' }
}
|
|
Should create a new bucket using the SDK's StorageHubClient:
test/test/suites/integration/solochain-evm/sdk-precompiles.test.ts#L316
AssertionError [ERR_ASSERTION]: MSP listBuckets should include the created bucket
at TestContext.<anonymous> (/home/runner/_work/storage-hub/storage-hub/test/suites/integration/solochain-evm/sdk-precompiles.test.ts:316:7)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async Test.run (node:internal/test_runner/test:1054:7)
at async Suite.processPendingSubtests (node:internal/test_runner/test:744:7) {
generatedMessage: false,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: '=='
}
|
|
Integration Tests: FullNet (1)
Process completed with exit code 1.
|
|
New MSP rejects move request due to low capacity:
test/test/suites/integration/msp/move-bucket-low-capacity.test.ts#L377
AssertionError [ERR_ASSERTION]: No events matching fileSystem.MoveBucketRejected
at assertEventPresent (/home/runner/_work/storage-hub/storage-hub/test/util/asserts.ts:166:3)
at TestContext.<anonymous> (/home/runner/_work/storage-hub/storage-hub/test/suites/integration/msp/move-bucket-low-capacity.test.ts:377:7)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async Test.run (node:internal/test_runner/test:1054:7)
at async Suite.processPendingSubtests (node:internal/test_runner/test:744:7) {
generatedMessage: false,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: '=='
}
|
|
New MSP rejects move request due to low capacity:
test/test/suites/integration/msp/move-bucket-low-capacity.test.ts#L377
AssertionError [ERR_ASSERTION]: No events matching fileSystem.MoveBucketRejected
at assertEventPresent (/home/runner/_work/storage-hub/storage-hub/test/util/asserts.ts:166:3)
at TestContext.<anonymous> (/home/runner/_work/storage-hub/storage-hub/test/suites/integration/msp/move-bucket-low-capacity.test.ts:377:7)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async Test.run (node:internal/test_runner/test:1054:7)
at async Suite.processPendingSubtests (node:internal/test_runner/test:744:7) {
generatedMessage: false,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: '=='
}
|
|
Integration Tests: FullNet (3)
Process completed with exit code 1.
|
|
Second file can no longer be downloaded from the MSP and BSP:
test/test/suites/integration/msp/duplicate-storage-request.test.ts#L440
AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:
waitFor({
lambda: async () => {
// Check file is NOT in BSP forest
const bspForestResult = await bspApi.rpc.storagehubclient.isFileInForest(
null,
file1.fileKey
);
if (bspForestResult.isTrue) {
return false;
}
// Check file is NOT in BSP file storage
const bspFileStorageResult = await bspApi.rpc.storagehubclient.isFileInFileStorage(
file1.fileKey
);
if (bspFileStorageResult.isFileFound) {
return false;
}
// Check file is NOT in MSP forest
const mspForestResult = await mspApi.rpc.storagehubclient.isFileInForest(
file2.bucketId,
file1.fileKey
);
if (mspForestResult.isTrue) {
return false;
}
// Check file is NOT in MSP file storage
const mspFileStorageResult = await mspApi.rpc.storagehubclient.isFileInFileStorage(
file1.fileKey
);
if (mspFileStorageResult.isFileFound) {
return false;
}
return true;
}
})
at TestContext.<anonymous> (/home/runner/_work/storage-hub/storage-hub/test/suites/integration/msp/duplicate-storage-request.test.ts:440:7)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async Test.run (node:internal/test_runner/test:1054:7)
at async Suite.processPendingSubtests (node:internal/test_runner/test:744:7) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: '=='
}
|
|
User deletes second file and Fisherman deletes it from Bucket's forest and BSP's forest:
test/test/suites/integration/msp/duplicate-storage-request.test.ts#L368
Error: Timeout waiting for fisherman to process User deletions after 25 seconds
at waitForFishermanBatchDeletions (/home/runner/_work/storage-hub/storage-hub/test/util/bspNet/fisherman.ts:503:17)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async Module.retryableWaitAndVerifyBatchDeletions (/home/runner/_work/storage-hub/storage-hub/test/util/bspNet/fisherman.ts:555:28)
at async TestContext.<anonymous> (/home/runner/_work/storage-hub/storage-hub/test/suites/integration/msp/duplicate-storage-request.test.ts:368:7)
at async Test.run (node:internal/test_runner/test:1054:7)
at async Suite.processPendingSubtests (node:internal/test_runner/test:744:7)
|
|
Second file can no longer be downloaded from the MSP and BSP:
test/test/suites/integration/msp/duplicate-storage-request.test.ts#L440
AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:
waitFor({
lambda: async () => {
// Check file is NOT in BSP forest
const bspForestResult = await bspApi.rpc.storagehubclient.isFileInForest(
null,
file1.fileKey
);
if (bspForestResult.isTrue) {
return false;
}
// Check file is NOT in BSP file storage
const bspFileStorageResult = await bspApi.rpc.storagehubclient.isFileInFileStorage(
file1.fileKey
);
if (bspFileStorageResult.isFileFound) {
return false;
}
// Check file is NOT in MSP forest
const mspForestResult = await mspApi.rpc.storagehubclient.isFileInForest(
file2.bucketId,
file1.fileKey
);
if (mspForestResult.isTrue) {
return false;
}
// Check file is NOT in MSP file storage
const mspFileStorageResult = await mspApi.rpc.storagehubclient.isFileInFileStorage(
file1.fileKey
);
if (mspFileStorageResult.isFileFound) {
return false;
}
return true;
}
})
at TestContext.<anonymous> (/home/runner/_work/storage-hub/storage-hub/test/suites/integration/msp/duplicate-storage-request.test.ts:440:7)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async Test.run (node:internal/test_runner/test:1054:7)
at async Suite.processPendingSubtests (node:internal/test_runner/test:744:7) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: '=='
}
|
|
User deletes second file and Fisherman deletes it from Bucket's forest and BSP's forest:
test/test/suites/integration/msp/duplicate-storage-request.test.ts#L368
Error: Timeout waiting for fisherman to process User deletions after 25 seconds
at waitForFishermanBatchDeletions (/home/runner/_work/storage-hub/storage-hub/test/util/bspNet/fisherman.ts:503:17)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async Module.retryableWaitAndVerifyBatchDeletions (/home/runner/_work/storage-hub/storage-hub/test/util/bspNet/fisherman.ts:555:28)
at async TestContext.<anonymous> (/home/runner/_work/storage-hub/storage-hub/test/suites/integration/msp/duplicate-storage-request.test.ts:368:7)
at async Test.run (node:internal/test_runner/test:1054:7)
at async Suite.processPendingSubtests (node:internal/test_runner/test:744:7)
|
|
Integration Tests: Fisherman
Process completed with exit code 1.
|
|
indexes BspConfirmStoppedStoring events:
test/test/suites/integration/fisherman/indexer-fishing.test.ts#L6
Error [RpcError]: 1014: Priority is too low: (215 vs 165): The transaction has too low priority to replace another transaction already in the pool.
at checkError (file:///home/runner/_work/storage-hub/storage-hub/node_modules/.pnpm/@PolkaDot[email protected]/node_modules/@polkadot/rpc-provider/coder/index.js:19:15)
at RpcCoder.decodeResponse (file:///home/runner/_work/storage-hub/storage-hub/node_modules/.pnpm/@PolkaDot[email protected]/node_modules/@polkadot/rpc-provider/coder/index.js:35:9)
at #onSocketMessageResult (file:///home/runner/_work/storage-hub/storage-hub/node_modules/.pnpm/@PolkaDot[email protected]/node_modules/@polkadot/rpc-provider/ws/index.js:429:40)
at #onSocketMessage (file:///home/runner/_work/storage-hub/storage-hub/node_modules/.pnpm/@PolkaDot[email protected]/node_modules/@polkadot/rpc-provider/ws/index.js:418:42)
at [nodejs.internal.kHybridDispatch] (node:internal/event_target:827:20)
at WebSocket.dispatchEvent (node:internal/event_target:762:26)
at fireEvent (node:internal/deps/undici/undici:11663:14)
at websocketMessageReceived (node:internal/deps/undici/undici:11685:7)
at ByteParser.run (node:internal/deps/undici/undici:12307:19)
at ByteParser._write (node:internal/deps/undici/undici:12200:14)
|
|
Integration Tests: Solochain EVM
Cache not found for keys: v0-rust-solochain_evm-Linux-x64-3d7017e8-0b1c9570, v0-rust-solochain_evm-Linux-x64-3d7017e8
|
|
sccache stats
0% - 0 hits, 0 misses, 0 errors
|
|
sccache stats
100% - 1092 hits, 0 misses, 0 errors
|
|
Node Tests: Solo Dev Node (1)
Total Tests: 5
Suites 📂: 1
Passed ✅: 5
Failed ❌: 0
Canceled 🚫: 0
Skipped ⏭️: 0
Todo 📝: 0
Duration 🕐: 14497.678ms
|
|
Node Tests: Solo Dev Node (2)
Total Tests: 3
Suites 📂: 1
Passed ✅: 3
Failed ❌: 0
Canceled 🚫: 0
Skipped ⏭️: 0
Todo 📝: 0
Duration 🕐: 14857.162ms
|
|
Integration Tests: User (2)
Total Tests: 14
Suites 📂: 2
Passed ✅: 14
Failed ❌: 0
Canceled 🚫: 0
Skipped ⏭️: 0
Todo 📝: 0
Duration 🕐: 64694.549ms
|
|
Integration Tests: FullNet (4)
Total Tests: 17
Suites 📂: 4
Passed ✅: 17
Failed ❌: 0
Canceled 🚫: 0
Skipped ⏭️: 0
Todo 📝: 0
Duration 🕐: 130000.786ms
|
|
Integration Tests: User (1)
Total Tests: 16
Suites 📂: 4
Passed ✅: 16
Failed ❌: 0
Canceled 🚫: 0
Skipped ⏭️: 0
Todo 📝: 0
Duration 🕐: 151392.854ms
|
|
Integration Tests: Backend (1)
Total Tests: 82
Suites 📂: 8
Passed ✅: 78
Failed ❌: 0
Canceled 🚫: 0
Skipped ⏭️: 4
Todo 📝: 0
Duration 🕐: 210196.626ms
|
|
Integration Tests: FullNet (2)
Total Tests: 20
Suites 📂: 5
Passed ✅: 20
Failed ❌: 0
Canceled 🚫: 0
Skipped ⏭️: 0
Todo 📝: 0
Duration 🕐: 229840.618ms
|
|
Integration Tests: Solochain EVM
Total Tests: 26
Suites 📂: 4
Passed ✅: 20
Failed ❌: 6
Canceled 🚫: 0
Skipped ⏭️: 0
Todo 📝: 0
Duration 🕐: 130607.813ms
|
|
Integration Tests: BSPNet (4)
Total Tests: 24
Suites 📂: 6
Passed ✅: 19
Failed ❌: 0
Canceled 🚫: 0
Skipped ⏭️: 5
Todo 📝: 0
Duration 🕐: 268120.896ms
|
|
Integration Tests: BSPNet (2)
Total Tests: 19
Suites 📂: 7
Passed ✅: 15
Failed ❌: 0
Canceled 🚫: 0
Skipped ⏭️: 4
Todo 📝: 0
Duration 🕐: 283336.795ms
|
|
Integration Tests: BSPNet (5)
Total Tests: 31
Suites 📂: 6
Passed ✅: 29
Failed ❌: 0
Canceled 🚫: 0
Skipped ⏭️: 2
Todo 📝: 0
Duration 🕐: 292297.155ms
|
|
Integration Tests: BSPNet (3)
Total Tests: 45
Suites 📂: 7
Passed ✅: 45
Failed ❌: 0
Canceled 🚫: 0
Skipped ⏭️: 0
Todo 📝: 0
Duration 🕐: 303089.030ms
|
|
Integration Tests: BSPNet (1)
Total Tests: 21
Suites 📂: 11
Passed ✅: 21
Failed ❌: 0
Canceled 🚫: 0
Skipped ⏭️: 0
Todo 📝: 0
Duration 🕐: 328907.985ms
|
|
Integration Tests: FullNet (1)
Total Tests: 28
Suites 📂: 8
Passed ✅: 26
Failed ❌: 2
Canceled 🚫: 0
Skipped ⏭️: 0
Todo 📝: 0
Duration 🕐: 356296.809ms
|
|
Integration Tests: FullNet (3)
Total Tests: 32
Suites 📂: 5
Passed ✅: 28
Failed ❌: 4
Canceled 🚫: 0
Skipped ⏭️: 0
Todo 📝: 0
Duration 🕐: 392255.064ms
|
|
Integration Tests: Fisherman
Total Tests: 38
Suites 📂: 6
Passed ✅: 37
Failed ❌: 1
Canceled 🚫: 0
Skipped ⏭️: 0
Todo 📝: 0
Duration 🕐: 735665.103ms
|
Artifacts
Produced during runtime
| Name | Size | Digest | |
|---|---|---|---|
|
fisherman-test-logs
|
127 KB |
sha256:c09fa779f586aa8d02c5ae422a83053bb061558e58f66fa9ace1b01aa59d4fb6
|
|
|
node
|
68.2 MB |
sha256:fbdca1ef696cf55b897ba2e234979e2df49577b8290e0693531473e764350a15
|
|
|
solochain-evm-test-logs
|
23.8 KB |
sha256:036a6a19cb13b0b29ba20c7255132ea7018b521a95d2cc5ee2827d14ffcbfaca
|
|
|
sp-test-logs-shard-1
|
16.7 KB |
sha256:4c8333d229a7456c7166e495e557441b1008b4cab57fe3a7bf8ba1725b7b908e
|
|
|
sp-test-logs-shard-3
|
23.2 KB |
sha256:f78a1463d5868e99a35825f66e8ef7292adc5457e5eee2dcf9efc66b669db5b8
|
|