refactor: ♻️ Uncouple HyperLiquid & Associated code (#69)

* first attempt

* refactor: ♻️ remove old references

* update ci

* typo

Author: timbrinded

.github/workflows/CI.yml

@@ -62,14 +62,38 @@ jobs:
       - *uv-sync
       - name: Run CLI with help flag
         run: uv run tq-oracle --help
-  
-  unittest:
+
+  unit-test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - *uv-install
+      - *uv-python
+      - *uv-sync
+      - run: uv run pytest
+
+  integration-test:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
       - *uv-install
       - *uv-python
       - *uv-sync
-      - name: Run unit tests with pytest
-        run: uv run pytest
+      - run: uv run pytest --run-integration
 
+  live-run:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - *uv-install
+      - *uv-python
+      - *uv-sync
+      - name: Run against eth test vault
+        run: |
+          uv run tq-oracle --network mainnet \
+                           --ignore-empty-vault \
+                           --ignore-timeout-check \
+                           --ignore-active-proposal-check \
+                           --log-level debug \
+                           --vault-rpc ${{ secrets.ETH_RPC }} \
+                             0xDbC81B33A23375A90c8Ba4039d5738CB6f56fE8d

.gitignore

@@ -28,6 +28,5 @@ CLAUDE.md
 
 .env
 
-test.toml
-config.toml
-tq-oracle.toml
+**/*.toml
+!py_project.toml

ARCHITECTURE.md

@@ -33,7 +33,7 @@ The CLI entry point uses Typer for command-line interface management. It:
 **Key Configuration:**
 
 - Network selection (mainnet, sepolia, base)
-- RPC endpoints (L1 and Hyperliquid)
+- RPC endpoints for vault network
 - Oracle and OracleHelper contract addresses
 - Safe multi-sig configuration
 - Dry-run vs production mode
@@ -55,7 +55,6 @@ The pipeline is the core orchestrator that sequences all oracle operations:
 Runs validation checks before processing TVL data to prevent race conditions and ensure data integrity:
 
 - **Safe State Check** (`check_adapters/safe_state.py`): TODO: Ensures no duplicate or pending reports
-- **CCTP Bridge Check** (`check_adapters/cctp_bridge.py`): Detects in-flight USDC transfers between L1 and Hyperliquid
 - **Timeout Check** (`check_adapters/timeout_check.py`): Verifies sufficient time has passed since last report
 - **Active Proposal Check** (`check_adapters/active_submit_report_proposal_check.py`): Checks for existing pending proposals
 
@@ -73,7 +72,6 @@ Discovers and aggregates asset holdings from multiple sources:
 **Supported Adapters** (`adapters/asset_adapters/`):
 
 - `IdleBalancesAdapter`: Checks for assets sat dormant in subvault
-- `HyperliquidAdapter`: Fetches portfolio value from Hyperliquid DEX
 
 #### 2.3 Pricing & Validation (`pipeline/pricing.py`)
 
@@ -172,7 +170,7 @@ sequenceDiagram
 ### Data Flow
 
 1. **Off-Chain Aggregation**:
-   - TQ Oracle queries multiple data sources (Hyperliquid API, L1 contracts, DEXs)
+   - TQ Oracle queries multiple data sources (vault chain contracts, DEXs, optional integrations)
    - Aggregates asset amounts across all subvaults
    - Fetches market prices from various sources
 
@@ -198,7 +196,8 @@ TQ Oracle uses a three-tier configuration system with precedence:
 
 - `vault_address`: Target vault to report on
 - `oracle_helper_address`: OracleHelper contract for price normalization
-- `vault_rpc` / `hl_rpc`: RPC endpoints for different chains (vault network and Hyperliquid)
+- `vault_rpc`: RPC endpoint for the vault network
+- `hl_rpc` (optional/disabled): Hyperliquid RPC endpoint reserved for future use
 - `safe_address`: Gnosis Safe for multi-sig submission
 - `private_key`: Signer key for proposing Safe transactions
 - `subvault_adapters`: Per-subvault adapter configuration
@@ -208,7 +207,7 @@ TQ Oracle uses a three-tier configuration system with precedence:
 Allows fine-grained control over which adapters run for each subvault via TOML configuration:
 
 - Specify target subvault address
-- Define which chain the subvault operates on (L1 or Hyperliquid)
+- Define which chain the subvault operates on (typically L1; Hyperliquid support requires re-enabling the integration)
 - List additional adapters to run for this subvault
 - Option to skip default idle balances check
 - Option to skip subvault existence validation

README.md

@@ -64,27 +64,17 @@ All configuration options can be set via CLI arguments, environment variables, o
 
 | CLI Option | Environment Variable | TOML Key | Default | Description |
 |------------|---------------------|-----------|---------|-------------|
-| `VAULT_ADDRESS` | - | `vault_address` | *required* | Vault contract address (positional argument) |
+| `vault_address` (argument) | `TQ_ORACLE_VAULT_ADDRESS` | `vault_address` | *required* | Vault contract address passed as positional argument |
 | `--config` `-c` | - | - | Auto-detect | Path to TOML configuration file |
-| `--oracle-helper-address` `-h` | - | `oracle_helper_address` | Auto (mainnet/testnet) | OracleHelper contract address |
-| `--vault-rpc` | `VAULT_RPC` | `vault_rpc` | Auto (mainnet/sepolia/base) | Vault network RPC endpoint |
-| `--hl-rpc` | `HL_EVM_RPC` | `hl_rpc` | Auto (mainnet/testnet) | Hyperliquid RPC endpoint |
-| `--l1-subvault-address` | `L1_SUBVAULT_ADDRESS` | `l1_subvault_address` | - | L1 subvault for CCTP monitoring |
-| `--hl-subvault-address` | `HL_SUBVAULT_ADDRESS` | `hl_subvault_address` | Vault address | Hyperliquid subvault address |
-| `--safe-address` `-s` | - | `safe_address` | - | Gnosis Safe address for multi-sig |
-| `--hyperliquid-env` | `HYPERLIQUID_ENV` | `hyperliquid_env` | `"mainnet"` | Hyperliquid environment (`"mainnet"` or `"testnet"`) |
-| `--cctp-env` | `CCTP_ENV` | `cctp_env` | `"mainnet"` | CCTP environment (`"mainnet"` or `"testnet"`) |
-| `--dry-run/--no-dry-run` | - | `dry_run` | `true` | Preview without sending transaction |
-| `--private-key` | `PRIVATE_KEY` | ❌ *Never use TOML* | - | Private key for signing (required with Safe on --no-dry-run) |
-| `--safe-key` | `SAFE_TRANSACTION_SERVICE_API_KEY` | ❌ *Never use TOML* | - | API key for Safe Transaction Service |
-| `--ignore-empty-vault` | - | `ignore_empty_vault` | `false` | Suppress errors for empty vaults |
-| `--ignore-timeout-check` | - | `ignore_timeout_check` | `false` | Allow forced submission bypassing timeout |
-| `--ignore-active-proposal-check` | - | `ignore_active_proposal_check` | `false` | Allow duplicate submissions |
-| `--pre-check-retries` | - | `pre_check_retries` | `3` | Number of pre-check retry attempts |
-| `--pre-check-timeout` | - | `pre_check_timeout` | `12.0` | Timeout between pre-check retries (seconds) |
-| - | - | `log_level` | `"INFO"` | Logging level (DEBUG, INFO, WARNING, ERROR, CRITICAL) |
-| - | - | `chainlink_price_warning_tolerance_percentage` | `0.5` | Price deviation warning threshold (%) |
-| - | - | `chainlink_price_failure_tolerance_percentage` | `1.0` | Price deviation failure threshold (%) |
+| `--network` `-n` | `TQ_ORACLE_NETWORK` | `network` | `"mainnet"` | Network to report on (`mainnet`, `sepolia`, `base`) |
+| `--block-number` | `TQ_ORACLE_BLOCK_NUMBER` | `block_number` | Latest block | Block number to snapshot vault state |
+| `--vault-rpc` | `TQ_ORACLE_VAULT_RPC` | `vault_rpc` | Network default | RPC endpoint for the selected vault network |
+| `--dry-run/--no-dry-run` | `TQ_ORACLE_DRY_RUN` | `dry_run` | `true` | Preview report without submitting a Safe transaction |
+| `--ignore-empty-vault/--require-nonempty-vault` | `TQ_ORACLE_IGNORE_EMPTY_VAULT` | `ignore_empty_vault` | `false` | Skip failure when vault holds zero assets |
+| `--ignore-timeout-check/--enforce-timeout-check/--enforce-timeout-check` | `TQ_ORACLE_IGNORE_TIMEOUT_CHECK` | `ignore_timeout_check` | `false` | Skip minimum interval guard between reports |
+| `--ignore-active-proposal-check/--enforce-active-proposal-check/--enforce-active-proposal-check` | `TQ_ORACLE_IGNORE_ACTIVE_PROPOSAL_CHECK` | `ignore_active_proposal_check` | `false` | Skip duplicate active proposal guard |
+| `--log-level` | `TQ_ORACLE_LOG_LEVEL` | `log_level` | `"INFO"` | Override logging verbosity (`TRACE`, `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`) |
+| `--show-config` | - | - | `false` | Dump effective configuration (with secrets redacted) and exit |
 
 #### TOML-Only Options (Not available via CLI)
 
@@ -94,6 +84,8 @@ All configuration options can be set via CLI arguments, environment variables, o
 | `rpc_max_concurrent_calls` | `5` | Maximum concurrent RPC connections |
 | `rpc_delay` | `0.15` | Delay between RPC calls (seconds) |
 | `rpc_jitter` | `0.10` | Random jitter for RPC delays (seconds) |
+| `price_warning_tolerance_percentage` | `0.5` | Price deviation warning threshold (%) |
+| `price_failure_tolerance_percentage` | `1.0` | Price deviation failure threshold (%) |
 
 ### Advanced Configuration: Subvault Adapters
 
@@ -102,15 +94,17 @@ TQ Oracle supports configuring multiple asset adapters per subvault address, all
 **Configuration via TOML:**
 
 - Specify target subvault address
-- Define which chain the subvault operates on (L1 or Hyperliquid)
+- Define which chain the subvault operates on (typically `vault_chain`)
 - List additional adapters to run for this subvault
 - Option to skip default idle balances check
 - Option to skip subvault existence validation
 
 **Available Asset Adapters:**
 
 - `idle_balances` - Checks for idle USDC balances not yet deployed
-- `hyperliquid` - Fetches portfolio value from Hyperliquid
+- `hyperliquid` *(disabled)* - See the restore checklist before re-enabling
+
+
 
 See `tq-oracle-example.toml` for complete configuration examples.
 
@@ -148,10 +142,15 @@ tq-oracle --network sepolia 0xYourVaultAddress
 tq-oracle --show-config
 ```
 
+**Increase verbosity for debugging:**
+
+```bash
+tq-oracle --log-level DEBUG
+```
+
 **Common Usage Patterns:**
 
 - **Dry-run on mainnet**: Preview report generation without submitting to chain
-- **Custom subvault**: Specify Hyperliquid subvault address for cross-chain asset tracking
 - **Testnet execution**: Run against testnet with Safe multi-sig for testing
 - **Pre-deployment testing**: Test with empty vaults using ignore flags
 
@@ -177,20 +176,20 @@ src/tq_oracle/
 Before processing TVL data, TQ Oracle runs automated pre-flight validation checks to ensure data integrity:
 
 - **Safe State Validation**: Ensures no duplicate or pending reports exist
-- **CCTP Bridge Detection**: Identifies in-flight USDC transfers between L1 and Hyperliquid
 - **Check Retry Logic**: Automatically retries failed checks with exponential backoff when recommended
+- **CCTP Bridge Detection** *(disabled)*: Re-enable via the checklist when cross-chain monitoring is required
 
-These checks prevent race conditions and ensure accurate TVL snapshots by detecting ongoing cross-chain transfers that could affect asset balances.
+These checks prevent race conditions and ensure accurate TVL snapshots by detecting ongoing cross-chain transfers that could affect asset balances. You can bypass individual guards when needed via the CLI flags `--ignore-empty-vault`, `--ignore-timeout-check/--enforce-timeout-check`, and `--ignore-active-proposal-check/--enforce-active-proposal-check`.
 
 ### Testing CCTP Bridge Detection
 
-A standalone test script is available in `scripts/check_cctp_inflight.py` to verify CCTP bridge in-flight detection with L1 and Hyperliquid subvault addresses.
+A standalone test script is available in `scripts/check_cctp_inflight.py` and can be used once the CCTP integration is re-enabled.
 
 ## Adding New Adapters
 
 ### Asset Adapters
 
-Asset adapters fetch asset holdings from specific protocols (e.g., Hyperliquid, Aave, Lido).
+Asset adapters fetch asset holdings from specific protocols (e.g., Aave, Lido).
 
 Quick overview:
 
@@ -223,8 +222,9 @@ Price validators cross-check prices from the main price adapters against referen
 5. **Write unit tests** in `tests/adapters/price_validators/`
 
 Validators respect tolerance thresholds configured in `settings.py`:
-- `chainlink_price_warning_tolerance_percentage` (default: 0.5%) - Issues warnings
-- `chainlink_price_failure_tolerance_percentage` (default: 1.0%) - Halts execution
+
+- `price_warning_tolerance_percentage` (default: 0.5%) - Issues warnings
+- `price_failure_tolerance_percentage` (default: 1.0%) - Halts execution
 
 ## Development
 
@@ -238,6 +238,5 @@ Validators respect tolerance thresholds configured in `settings.py`:
 ## External Links
 
 - `flexible-vaults` [repo](https://github.com/mellow-finance/flexible-vaults)
-- Hyperliquid API [documentation](https://hyperliquid.gitbook.io/hyperliquid-docs/for-developers/api/info-endpoint)
 - `cctp-v2` [contracts](https://github.com/circlefin/evm-cctp-contracts/tree/master/src/v2)
 - DeBridge [contracts](https://github.com/debridge-finance/dln-contracts/tree/main/contracts/DLN)

docs/hyperliquid-cctp-restore-checklist.md

@@ -0,0 +1,32 @@
+# Hyperliquid & CCTP Restore Checklist
+
+Use this checklist when you are ready to ship Hyperliquid support and the CCTP pre-check again. Work through every section before exposing the functionality.
+
+## 1. Dependencies
+
+- [ ] Add the hyperliquid sdk back to `pyproject.toml` and run **uv sync** to update `uv.lock`.
+- [ ] Confirm RPC providers and API endpoints for the target Hyperliquid environment(s).
+
+## 2. Configuration & CLI Surface
+
+- [ ] Restore Hyperliquid/CCTP fields in `OracleSettings` (environment toggles, RPC/subvault fields, token messenger settings).
+- [ ] Reinstate logging/telemetry in `pipeline/run.py` that records Hyperliquid and CCTP settings.
+- [ ] Reintroduce CLI flags for Hyperliquid/CCTP in `main.py` (`--hyperliquid-env`, `--hl-rpc`, `--hl-block-number`, `--cctp-env`).
+- [ ] Update documentation (README/ARCHITECTURE) and sample configs once fields exist again.
+
+## 3. Runtime Wiring
+
+- [ ] Re-register `HyperliquidAdapter` in `src/tq_oracle/adapters/asset_adapters/__init__.py`.
+- [ ] Allow `collect_assets` to schedule Hyperliquid adapters instead of skipping disabled entries.
+- [ ] Ensure idle balances on Hyperliquid still function (`IdleBalancesAdapter` with `chain="hyperliquid"`).
+
+## 4. CCTP Pre-Check
+
+- [ ] Re-enable `CCTPBridgeAdapter` in `src/tq_oracle/adapters/check_adapters/__init__.py`.
+- [ ] Validate CCTP config fields (token messenger address, hl/l1 subvault addresses) are surfaced and documented.
+- [ ] Confirm `scripts/check_cctp_inflight.py` still works end-to-end.
+
+## 5. Validation & Monitoring
+
+- [ ] Re-run unit/integration tests covering Hyperliquid + CCTP (`tests/adapters/asset_adapters/test_hyperliquid.py`, `tests/adapters/check_adapters/test_cctp_bridge.py`, pipeline smoke tests).
+- [ ] Execute manual dry-runs against staging vaults to verify NAV + pre-check behaviour.

pyproject.toml

@@ -7,7 +7,6 @@ requires-python = ">=3.11"
 dependencies = [
     "backoff>=2.2.1",
     "dotenv>=0.9.9",
-    "hyperliquid-python-sdk>=0.19.0",
     "pydantic>=2.6",
     "pydantic-settings>=2.2",
     "requests>=2.32.5",