feat: RocksDB Column Family Migration Framework

[snowmead]

Summary

Adds a versioned migration framework for managing RocksDB column family schema changes. This enables safe removal of deprecated column families when upgrading existing databases, ensuring backward compatibility with older node installations.

Design

RocksDB requires all existing column families to be specified when opening a database. The migration system discovers existing CFs, opens the database, then drops deprecated ones via versioned migrations.

How it works:

1. Discover existing column families via DB::list_cf()
2. Open database with existing + current CFs
3. Run pending migrations to drop deprecated CFs
4. Update schema version (tracked in __schema_version__ CF)

Safety features:

• Downgrade prevention (rejects newer schema versions)
• Validates migration sequence (no gaps/duplicates)
• Cleanup pass on startup handles partial failures

New Module Structure

client/common/src/rocksdb/
├── mod.rs          # Module exports and documentation
├── database.rs     # Database opening functions (open_db, open_db_with_migrations)
├── migrations.rs   # Migration trait, MigrationRunner, schema version tracking
└── tests.rs        # Comprehensive test coverage for migration scenarios

Notable Changes

• New Migration trait and MigrationRunner in client/common/src/rocksdb/
• TypedRocksDB::open_with_migrations() for stores needing migrations
• TypedRocksDB::open() for stores without migrations
• Updated BlockchainServiceStateStore, DownloadStateStore, BspPeerManagerStore

Test Coverage

Tests use temporary RocksDB instances to verify migration functionality including schema version tracking, CF dropping, and error handling.

Adding New Migrations

// 1. Implement the Migration trait
pub struct MyStoreV2Migration;

impl Migration for MyStoreV2Migration {
    fn version(&self) -> u32 { 2 }
    fn deprecated_column_families(&self) -> &'static [&'static str] {
        &["old_cf_to_remove"]
    }
    fn description(&self) -> &'static str { "Remove old_cf_to_remove" }
}

// 2. Register in your store's migrations function
pub fn my_store_migrations() -> Vec<Box<dyn Migration>> {
    vec![Box::new(MyStoreV1Migration), Box::new(MyStoreV2Migration)]
}

// 3. Open with migrations
let db = TypedRocksDB::open_with_migrations(&path, &CURRENT_CFS, my_store_migrations())?;

⚠️ Breaking Changes ⚠️

• Short description
  A new v1 migration will be applied to the blockchain service state store to drop deprecated MSP respond storage request column families:
  • pending_msp_respond_storage_request
  • pending_msp_respond_storage_request_left_index
  • pending_msp_respond_storage_request_right_index
• Who is affected
  • MSP node runners.
• Suggested code changes
  No code changes required. The migration runs automatically on startup.