feat(fisherman): 🌎 batch file deletions #489
snowmeadnetwork.yml
on: pull_request
setup
8s
Push node Docker image to Docker Hub
50s
Network Test: Zombienet (Parachain + Relay)
3m 14s
Network Test: Zombienet CLI
2m 6s
Integration Tests: Fisherman
13m 52s
Check Rust/TS bindings are up to date
1m 16s
Matrix: Integration Tests: BSPNet
Matrix: Node Tests: Solo Dev Node
Matrix: Integration Tests: FullNet
Matrix: Integration Tests: User
Push backend Docker image to Docker Hub
0s
Integration Tests: Solochain EVM
4m 34s
Matrix: Integration Tests: Backend
Annotations
16 errors, 1 warning, and 18 notices
|
Integration Tests: Solochain EVM
Process completed with exit code 1.
|
|
Should fetch payment streams using the SDK's MspClient:
test/test/suites/integration/solochain-evm/sdk-precompiles.test.ts#L481
AssertionError [ERR_ASSERTION]: SDK did not return a payment stream for the expected MSP
at TestContext.<anonymous> (/home/runner/_work/storage-hub/storage-hub/test/suites/integration/solochain-evm/sdk-precompiles.test.ts:481:7)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async Test.run (node:internal/test_runner/test:1054:7)
at async Suite.processPendingSubtests (node:internal/test_runner/test:744:7) {
generatedMessage: false,
code: 'ERR_ASSERTION',
actual: undefined,
expected: true,
operator: '=='
}
|
|
Should upload the file to the MSP through the backend using the SDK's StorageHubClient:
test/test/suites/integration/solochain-evm/sdk-precompiles.test.ts#L458
Error [HttpError]: HTTP 500 for GET http://127.0.0.1:8080/buckets/0xb1bf58ff2c2b53bdaddc9279116e0af3bdd77b7a656aaf65a0446242c769e1a0/files
at H.request (/home/runner/_work/storage-hub/storage-hub/sdk/core/dist/index.node.js:1:242583)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async TestContext.<anonymous> (/home/runner/_work/storage-hub/storage-hub/test/suites/integration/solochain-evm/sdk-precompiles.test.ts:458:24)
at async Test.run (node:internal/test_runner/test:1054:7)
at async Suite.processPendingSubtests (node:internal/test_runner/test:744:7) {
status: 500,
body: { error: 'Database error: Record not found' }
}
|
|
Should create a new bucket using the SDK's StorageHubClient:
test/test/suites/integration/solochain-evm/sdk-precompiles.test.ts#L316
AssertionError [ERR_ASSERTION]: MSP listBuckets should include the created bucket
at TestContext.<anonymous> (/home/runner/_work/storage-hub/storage-hub/test/suites/integration/solochain-evm/sdk-precompiles.test.ts:316:7)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async Test.run (node:internal/test_runner/test:1054:7)
at async Suite.processPendingSubtests (node:internal/test_runner/test:744:7) {
generatedMessage: false,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: '=='
}
|
|
Should fetch payment streams using the SDK's MspClient:
test/test/suites/integration/solochain-evm/sdk-precompiles.test.ts#L481
AssertionError [ERR_ASSERTION]: SDK did not return a payment stream for the expected MSP
at TestContext.<anonymous> (/home/runner/_work/storage-hub/storage-hub/test/suites/integration/solochain-evm/sdk-precompiles.test.ts:481:7)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async Test.run (node:internal/test_runner/test:1054:7)
at async Suite.processPendingSubtests (node:internal/test_runner/test:744:7) {
generatedMessage: false,
code: 'ERR_ASSERTION',
actual: undefined,
expected: true,
operator: '=='
}
|
|
Should upload the file to the MSP through the backend using the SDK's StorageHubClient:
test/test/suites/integration/solochain-evm/sdk-precompiles.test.ts#L458
Error [HttpError]: HTTP 500 for GET http://127.0.0.1:8080/buckets/0xb1bf58ff2c2b53bdaddc9279116e0af3bdd77b7a656aaf65a0446242c769e1a0/files
at H.request (/home/runner/_work/storage-hub/storage-hub/sdk/core/dist/index.node.js:1:242583)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async TestContext.<anonymous> (/home/runner/_work/storage-hub/storage-hub/test/suites/integration/solochain-evm/sdk-precompiles.test.ts:458:24)
at async Test.run (node:internal/test_runner/test:1054:7)
at async Suite.processPendingSubtests (node:internal/test_runner/test:744:7) {
status: 500,
body: { error: 'Database error: Record not found' }
}
|
|
Should create a new bucket using the SDK's StorageHubClient:
test/test/suites/integration/solochain-evm/sdk-precompiles.test.ts#L316
AssertionError [ERR_ASSERTION]: MSP listBuckets should include the created bucket
at TestContext.<anonymous> (/home/runner/_work/storage-hub/storage-hub/test/suites/integration/solochain-evm/sdk-precompiles.test.ts:316:7)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async Test.run (node:internal/test_runner/test:1054:7)
at async Suite.processPendingSubtests (node:internal/test_runner/test:744:7) {
generatedMessage: false,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: '=='
}
|
|
Integration Tests: BSPNet (1)
Process completed with exit code 1.
|
|
BSPNet: BSPNet: Single BSP multi-volunteers (RocksDB):
test/test/util/bspNet/testrunner.ts#L1
Error: Timeout of 15000ms exceeded while waiting for log 💤 Idle
at Timeout.<anonymous> (/home/runner/_work/storage-hub/storage-hub/test/util/bspNet/docker.ts:605:15)
at listOnTimeout (node:internal/timers:608:17)
at process.processTimers (node:internal/timers:543:7)
|
|
BSP volunteers and confirms storing multiple files with batching:
test/test/suites/integration/bsp/single-volunteer.test.ts#L1
'test did not finish before its parent and was cancelled'
|
|
Network launches and can be queried:
test/test/suites/integration/bsp/single-volunteer.test.ts#L1
'test did not finish before its parent and was cancelled'
|
|
Integration Tests: FullNet (3)
Process completed with exit code 1.
|
|
Second file can no longer be downloaded from the MSP and BSP:
test/test/suites/integration/msp/duplicate-storage-request.test.ts#L440
AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:
waitFor({
lambda: async () => {
// Check file is NOT in BSP forest
const bspForestResult = await bspApi.rpc.storagehubclient.isFileInForest(
null,
file1.fileKey
);
if (bspForestResult.isTrue) {
return false;
}
// Check file is NOT in BSP file storage
const bspFileStorageResult = await bspApi.rpc.storagehubclient.isFileInFileStorage(
file1.fileKey
);
if (bspFileStorageResult.isFileFound) {
return false;
}
// Check file is NOT in MSP forest
const mspForestResult = await mspApi.rpc.storagehubclient.isFileInForest(
file2.bucketId,
file1.fileKey
);
if (mspForestResult.isTrue) {
return false;
}
// Check file is NOT in MSP file storage
const mspFileStorageResult = await mspApi.rpc.storagehubclient.isFileInFileStorage(
file1.fileKey
);
if (mspFileStorageResult.isFileFound) {
return false;
}
return true;
}
})
at TestContext.<anonymous> (/home/runner/_work/storage-hub/storage-hub/test/suites/integration/msp/duplicate-storage-request.test.ts:440:7)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async Test.run (node:internal/test_runner/test:1054:7)
at async Suite.processPendingSubtests (node:internal/test_runner/test:744:7) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: '=='
}
|
|
User deletes second file and Fisherman deletes it from Bucket's forest and BSP's forest:
test/test/suites/integration/msp/duplicate-storage-request.test.ts#L368
Error: Timeout waiting for fisherman to process User deletions after 25 seconds
at waitForFishermanBatchDeletions (/home/runner/_work/storage-hub/storage-hub/test/util/bspNet/fisherman.ts:503:17)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async Module.retryableWaitAndVerifyBatchDeletions (/home/runner/_work/storage-hub/storage-hub/test/util/bspNet/fisherman.ts:555:28)
at async TestContext.<anonymous> (/home/runner/_work/storage-hub/storage-hub/test/suites/integration/msp/duplicate-storage-request.test.ts:368:7)
at async Test.run (node:internal/test_runner/test:1054:7)
at async Suite.processPendingSubtests (node:internal/test_runner/test:744:7)
|
|
Second file can no longer be downloaded from the MSP and BSP:
test/test/suites/integration/msp/duplicate-storage-request.test.ts#L440
AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:
waitFor({
lambda: async () => {
// Check file is NOT in BSP forest
const bspForestResult = await bspApi.rpc.storagehubclient.isFileInForest(
null,
file1.fileKey
);
if (bspForestResult.isTrue) {
return false;
}
// Check file is NOT in BSP file storage
const bspFileStorageResult = await bspApi.rpc.storagehubclient.isFileInFileStorage(
file1.fileKey
);
if (bspFileStorageResult.isFileFound) {
return false;
}
// Check file is NOT in MSP forest
const mspForestResult = await mspApi.rpc.storagehubclient.isFileInForest(
file2.bucketId,
file1.fileKey
);
if (mspForestResult.isTrue) {
return false;
}
// Check file is NOT in MSP file storage
const mspFileStorageResult = await mspApi.rpc.storagehubclient.isFileInFileStorage(
file1.fileKey
);
if (mspFileStorageResult.isFileFound) {
return false;
}
return true;
}
})
at TestContext.<anonymous> (/home/runner/_work/storage-hub/storage-hub/test/suites/integration/msp/duplicate-storage-request.test.ts:440:7)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async Test.run (node:internal/test_runner/test:1054:7)
at async Suite.processPendingSubtests (node:internal/test_runner/test:744:7) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: '=='
}
|
|
User deletes second file and Fisherman deletes it from Bucket's forest and BSP's forest:
test/test/suites/integration/msp/duplicate-storage-request.test.ts#L368
Error: Timeout waiting for fisherman to process User deletions after 25 seconds
at waitForFishermanBatchDeletions (/home/runner/_work/storage-hub/storage-hub/test/util/bspNet/fisherman.ts:503:17)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async Module.retryableWaitAndVerifyBatchDeletions (/home/runner/_work/storage-hub/storage-hub/test/util/bspNet/fisherman.ts:555:28)
at async TestContext.<anonymous> (/home/runner/_work/storage-hub/storage-hub/test/suites/integration/msp/duplicate-storage-request.test.ts:368:7)
at async Test.run (node:internal/test_runner/test:1054:7)
at async Suite.processPendingSubtests (node:internal/test_runner/test:744:7)
|
|
Integration Tests: Solochain EVM
Cache not found for keys: v0-rust-solochain_evm-Linux-x64-3d7017e8-0b1c9570, v0-rust-solochain_evm-Linux-x64-3d7017e8
|
|
sccache stats
0% - 0 hits, 0 misses, 0 errors
|
|
sccache stats
100% - 1091 hits, 1 misses, 0 errors
|
|
Node Tests: Solo Dev Node (2)
Total Tests: 3
Suites 📂: 1
Passed ✅: 3
Failed ❌: 0
Canceled 🚫: 0
Skipped ⏭️: 0
Todo 📝: 0
Duration 🕐: 12069.525ms
|
|
Node Tests: Solo Dev Node (1)
Total Tests: 5
Suites 📂: 1
Passed ✅: 5
Failed ❌: 0
Canceled 🚫: 0
Skipped ⏭️: 0
Todo 📝: 0
Duration 🕐: 14151.820ms
|
|
Integration Tests: User (2)
Total Tests: 14
Suites 📂: 2
Passed ✅: 14
Failed ❌: 0
Canceled 🚫: 0
Skipped ⏭️: 0
Todo 📝: 0
Duration 🕐: 61049.492ms
|
|
Integration Tests: User (1)
Total Tests: 16
Suites 📂: 4
Passed ✅: 16
Failed ❌: 0
Canceled 🚫: 0
Skipped ⏭️: 0
Todo 📝: 0
Duration 🕐: 133413.120ms
|
|
Integration Tests: FullNet (4)
Total Tests: 17
Suites 📂: 4
Passed ✅: 17
Failed ❌: 0
Canceled 🚫: 0
Skipped ⏭️: 0
Todo 📝: 0
Duration 🕐: 131708.209ms
|
|
Integration Tests: Solochain EVM
Total Tests: 26
Suites 📂: 4
Passed ✅: 20
Failed ❌: 6
Canceled 🚫: 0
Skipped ⏭️: 0
Todo 📝: 0
Duration 🕐: 113236.654ms
|
|
Integration Tests: Backend (1)
Total Tests: 82
Suites 📂: 8
Passed ✅: 78
Failed ❌: 0
Canceled 🚫: 0
Skipped ⏭️: 4
Todo 📝: 0
Duration 🕐: 214950.527ms
|
|
Integration Tests: BSPNet (4)
Total Tests: 24
Suites 📂: 6
Passed ✅: 19
Failed ❌: 0
Canceled 🚫: 0
Skipped ⏭️: 5
Todo 📝: 0
Duration 🕐: 240956.462ms
|
|
Integration Tests: FullNet (2)
Total Tests: 20
Suites 📂: 5
Passed ✅: 20
Failed ❌: 0
Canceled 🚫: 0
Skipped ⏭️: 0
Todo 📝: 0
Duration 🕐: 244475.143ms
|
|
Integration Tests: BSPNet (2)
Total Tests: 19
Suites 📂: 7
Passed ✅: 15
Failed ❌: 0
Canceled 🚫: 0
Skipped ⏭️: 4
Todo 📝: 0
Duration 🕐: 278513.082ms
|
|
Integration Tests: BSPNet (3)
Total Tests: 45
Suites 📂: 7
Passed ✅: 45
Failed ❌: 0
Canceled 🚫: 0
Skipped ⏭️: 0
Todo 📝: 0
Duration 🕐: 290946.235ms
|
|
Integration Tests: BSPNet (5)
Total Tests: 31
Suites 📂: 6
Passed ✅: 29
Failed ❌: 0
Canceled 🚫: 0
Skipped ⏭️: 2
Todo 📝: 0
Duration 🕐: 319290.361ms
|
|
Integration Tests: BSPNet (1)
Total Tests: 21
Suites 📂: 11
Passed ✅: 19
Failed ❌: 0
Canceled 🚫: 2
Skipped ⏭️: 0
Todo 📝: 0
Duration 🕐: 344442.755ms
|
|
Integration Tests: FullNet (1)
Total Tests: 28
Suites 📂: 8
Passed ✅: 28
Failed ❌: 0
Canceled 🚫: 0
Skipped ⏭️: 0
Todo 📝: 0
Duration 🕐: 376243.766ms
|
|
Integration Tests: FullNet (3)
Total Tests: 32
Suites 📂: 5
Passed ✅: 28
Failed ❌: 4
Canceled 🚫: 0
Skipped ⏭️: 0
Todo 📝: 0
Duration 🕐: 427812.798ms
|
|
Integration Tests: Fisherman
Total Tests: 38
Suites 📂: 6
Passed ✅: 38
Failed ❌: 0
Canceled 🚫: 0
Skipped ⏭️: 0
Todo 📝: 0
Duration 🕐: 740339.483ms
|
Artifacts
Produced during runtime
| Name | Size | Digest | |
|---|---|---|---|
|
bsp-test-logs-shard-1
|
2.43 MB |
sha256:a9db907f039b9733b343def44665876d08863e600c64a44543f7efcda7d8bf1b
|
|
|
node
|
68.2 MB |
sha256:7e0625329d782212413856dda688c64c273923f6e1ebcfc09a3b99010f859eaf
|
|
|
solochain-evm-test-logs
|
23.9 KB |
sha256:dfffef4a38240bd3dd30ed738d91bf6df27fa4fb929adac61a94be750a3d4bb0
|
|
|
sp-test-logs-shard-3
|
23.2 KB |
sha256:4fdc55809ecb2dad1f3d4982ee39ffac0a4d569c3aa1709a3c3c2aef41240a26
|
|