StorageHub v0.2.3

Summary

StorageHub v0.2.3 includes a hotfix for how the indexer handles the deletion of files, preventing it if the file to delete currently has an open storage request on-chain. It also includes extra safeguards in the whole deletion process to avoid inconsistent or erroneous states, such as a BSP confirming to store a file that a fisherman has already deleted.

Components

• Client code: v0.2.3
• Pallets code: v0.2.3
• Runtime code: v0.2.3 (spec_name/spec_version: parachain 1, solochain-evm 1)
• SH Backend Docker image: v0.2.3 (image: ghcr.io/<org>/storage-hub-msp-backend:v0.2.3)
• SH SDK (npm): v0.3.4 (@storagehub-sdk/core, @storagehub-sdk/msp-client)
• types-bundle/api-augment (npm): @storagehub/types-bundle v0.2.8, @storagehub/api-augment v0.2.11

Changes since last tag

Base: 5e53d4c3e6b6d9524968c9149f9542a99ca0e72d

• Highlights:
  • Fix critical issue with indexer: the indexer will no longer delete files from its DB that currently have an open storage request on-chain. This avoid an issue where the indexer would get stuck if a BSP confirm or MSP accept came for a on-chain storage request for a file key that a fisherman has already finished up deleting.
  • Extra safeguards in the runtime: The runtime now also protects against this sort of inconsistencies, by requiring specific conditions in order to issue a new storage request or request the deletion of a file.
• Full diff: 5e53d4c...527f303
• PRs included:
  • fix: 🚑 indexer handling of file deletion (#596)

⚠️ Breaking Changes ⚠️

• #596: The indexer DB has two new possible Steps that a file can be in: revoked and rejected. The backend (which interacts with the indexer DB) was updated to handle the two new steps in its FileStatus model, and the SDK as well since it receives a FileStatus from the backend.

Client/Indexer

• Behaviour changes:
  • Graceful recovery for missing file records: The indexer no longer crashes if it doesn't find a file record during BspConfirmedStoring or MspAcceptedStorageRequest events. Instead, it recreates the file record from the event metadata. This is a recovery mechanism for edge cases where a file was deleted from the DB but a storage request was still present on-chain.
  • Protect files with active storage requests from deletion: The indexer will no longer delete file records that have an active storage request (step = Requested and no deletion status). This prevents race conditions where deletion events arrive before confirmation events, avoiding situations where the indexer would get stuck.
  • New file storage request steps: Added two new FileStorageRequestStep values: Revoked (step 3) and Rejected (step 4). Files are now marked with these steps when handling StorageRequestRevoked and MspRejectedStorageRequest events respectively, before attempting orphan cleanup.

Runtime

• Block new storage requests for files being cleaned up: A new storage request cannot be issued if there's an existing IncompleteStorageRequest in storage for that file key. Users must wait until the fisherman finishes cleaning up the previous storage request before issuing a new one.
• Block file deletion when storage request is active: Users cannot request a file deletion if there's either an open storage request for that file key or an IncompleteStorageRequest exists for it. For the former, users should call revoke_storage_request instead, for the latter they should wait until the fisherman finishes cleaning up the previous storage request.
• Handle open storage requests during deletion: If a storage request is open for a file key that the fisherman is deleting, the runtime now issues an IncompleteStorageRequest for it and cleans it up properly.

Backend

• New FileStatus values: Updated the FileStatus enum in the FileInfo model to include two new statuses: Revoked (when user explicitly revoked a storage request) and Rejected (when MSP rejected a storage request).

SDK

• Updated FileStatus type: The FileStatus type in @storagehub-sdk/msp-client now includes "revoked" and "rejected" as possible values, in addition to "inProgress", "ready", "expired", and "deletionInProgress".

Versions

• Polkadot SDK: polkadot-stable2412-6
• Rust: 1.87 (from rust-toolchain.toml)

Compatibility

• SH Backend v0.2.3 → compatible with pallets/runtime v0.2.3 and client v0.2.3 (all built from this release).
• SDK v0.3.4 → compatible with backend v0.2.3, client v0.2.3, and pallets/runtime v0.2.3.

Upgrade Guide

• #596 – handling the two new FileStatus that a file can be in:

The indexer DB has two new possible Steps that a file can be in: revoked and rejected. The backend (which interacts with the indexer DB) was updated to handle the two new steps in its FileStatus model, and the SDK as well since it receives a FileStatus from the backend.

• Who is affected

  • Backend API consumers calling /buckets/{bucket_id}/info/{file_key} since they now have two extra FileStatus to handle.
  • SDK users calling MspClient.files.getFileInfo(bucketId, fileKey) or MspClient.buckets.getFiles(bucketId, options?), since the files returned by these methods have these two new statuses to handle.

• Changes required

  • Backend API: If you have any logic that depends on the FileStatus returned when calling /buckets/{bucket_id}/info/{file_key}, handle the two new statuses.

  • SDK - MspClient: Any logic that uses the getFileInfo and getFiles endpoints should handle the two new statuses.

  • Suggested code changes: See PR description for instructions and code snippets.