docs: ✍️ Update documentation (#133)

* update readme

* updated architecture

* add diagram

* format

Author: timbrinded

ARCHITECTURE.md

@@ -24,12 +24,14 @@ flowchart TD
 ## Core Components
 
 ### CLI & Configuration (`src/tq_oracle/main.py`, `settings.py`)
+
 - Typer command parses CLI flags, applies overrides, and instantiates `OracleSettings`.
 - `OracleSettings` is the single truth for configuration: CLI args > environment/`.env` > TOML (`tq-oracle.toml` or `$HOME/.config/tq-oracle/config.toml`). Secrets are rejected from TOML and wrapped in `SecretStr`.
 - Network defaults come from `constants.py` (RPC URLs, OracleHelper contracts, and canonical asset maps for Mainnet, Sepolia, Base). Missing RPC/OracleHelper values are filled automatically; block numbers are resolved via Web3 when absent.
 - The CLI enforces required params (vault address, RPC, Safe creds when `--no-dry-run`) and exposes `--show-config` for redacted inspection before dispatching to `pipeline.run_report()`.
 
 ### Pipeline Orchestration (`pipeline/run.py`)
+
 `run_report()` builds a `PipelineContext` and executes the pipeline sequentially:
 1. **Base asset discovery** – `_discover_base_asset()` walks Vault → FeeManager contracts on the target chain to retrieve the canonical base asset used for pricing.
 2. **Preflight** – `pipeline/preflight.py` runs adapters with backoff/retry to ensure it is safe to proceed.
@@ -38,24 +40,33 @@ flowchart TD
 5. **Report + publish** – packages results and either prints or submits to a Safe.
 
 ### Preflight Checks (`checks/pre_checks.py`, `adapters/check_adapters/*`)
-- **Safe state**: warns/blocks if no Safe configured or if duplicate/pending oracle reports exist (`safe_state.py`, `active_submit_report_proposal_check.py`).
+
+- **Active proposal check**: blocks if a duplicate or pending oracle report already exists on the Safe (`active_submit_report_proposal_check.py`); can be bypassed via `--ignore-active-proposal-check`.
 - **Timeout enforcement**: queries the Oracle contract for last report timestamps and security params; can be bypassed via `--ignore-timeout-check` (`timeout_check.py`).
 - Each adapter returns a `CheckResult`, and `run_preflight()` retries failures up to `pre_check_retries` using `backoff` unless the adapter signals `retry_recommended=False`.
 
 ### Asset Collection (`pipeline/assets.py`, `adapters/asset_adapters/*`)
+
 - Fetches subvault addresses via Vault ABI calls, validates `subvault_adapters` config entries, and optionally allows synthetic addresses when `skip_subvault_existence_check` is set.
-- Runs the default `IdleBalancesAdapter` across the vault and every subvault (unless skipped per-subvault) to gather on-chain balances for supported assets defined in `OracleSettings.assets`.
-- Additional adapters can be configured per subvault; each adapter inherits from `BaseAssetAdapter` and is registered in `ADAPTER_REGISTRY`.
+- Three default adapters run automatically based on configuration:
+  - `IdleBalancesAdapter` – gathers on-chain token balances across vault and all subvaults (skippable per-subvault via `skip_idle_balances`).
+  - `StrETHAdapter` – fetches strETH positions (skippable via `skip_streth`).
+  - `StakeWiseAdapter` – fetches StakeWise vault positions when `stakewise_vault_addresses` is configured.
+- Additional adapters can be configured per subvault via `additional_adapters`; each adapter inherits from `BaseAssetAdapter` and is registered in `ADAPTER_REGISTRY`.
 - Results are combined asynchronously and folded into `AggregatedAssets` via `processors/asset_aggregator.py`.
 
 ### Pricing & Validation (`pipeline/pricing.py`, `adapters/price_adapters/*`, `adapters/price_validators/pyth.py`)
-- Price adapters are instantiated from `PRICE_ADAPTERS` (currently `CowSwapAdapter` for general assets and `ETHAdapter` for ETH/WETH;).
+
+- Price adapters are instantiated from `PRICE_ADAPTERS`:
+  - `CowSwapAdapter` – fetches prices from CoW Protocol for general assets.
+  - `ETHAdapter` – handles ETH/WETH pricing.
 - Each adapter updates a shared `PriceData` accumulator keyed by asset address (base asset must already be known).
-- `run_price_validations()` invokes validators from `PRICE_VALIDATORS`. The active `PythValidator` re-fetches prices through the Pyth Hermes API, compares deviations against configurable warning/failure tolerances, and can be disabled via `pyth_enabled=False`.
+- `run_price_validations()` invokes validators from `PRICE_VALIDATORS`. The active `PythValidator` re-fetches prices through the Pyth Hermes API and compares deviations against configurable warning/failure tolerances (`price_warning_tolerance_percentage`, `price_failure_tolerance_percentage`).
 - `processors/total_assets.calculate_total_assets()` multiplies balances by prices (18-decimal math) and raises if any asset lacks a quote.
 - `processors/oracle_helper.derive_final_prices()` submits the total asset figure plus encoded prices to the OracleHelper contract to obtain finalized per-asset values (respecting `ignore_empty_vault`).
 
 ### Reporting & Publishing (`report/*`)
+
 - `report/generator.py` converts context data into an `OracleReport` dataclass.
 - `report/encoder.py` encodes `submitReports(Report[] reports)` calldata sorted by address and forces the base asset price to zero, mirroring the Solidity helper expectations.
 - `report/publisher.py`:
@@ -72,6 +83,7 @@ Key `OracleSettings` fields:
 - Pricing knobs: `pyth_*` settings plus warning/failure tolerances.
 
 ### Subvault Adapter Configuration
+
 Each entry under `subvault_adapters` may specify:
 - `subvault_address`: Target contract (checksummed matching Vault discovery unless `skip_subvault_existence_check` is true).
 - `skip_idle_balances`: Exclude default idle balance scan for that address.
@@ -98,6 +110,105 @@ sequenceDiagram
     end
 ```
 
+### How Mellow Vaults Work
+
+Mellow flexible vaults are modular yield-bearing vaults where user deposits flow through a layered architecture:
+
+```mermaid
+flowchart TB
+    subgraph Vault
+        SM[ShareManager<br/>mint/burn]
+        FM[FeeManager<br/>perf/protocol fees]
+        RM[RiskManager<br/>TVL limits]
+        Q[Deposit/Redeem Queues<br/>batch requests, settle at oracle price]
+    end
+    Vault --> SV1[Subvault<br/>EigenLayer]
+    Vault --> SV2[Subvault<br/>Symbiotic]
+    Vault --> SV3[Subvault<br/>Aave]
+```
+
+**Deposit-to-share lifecycle:**
+
+```mermaid
+sequenceDiagram
+    participant User
+    participant DepositQueue
+    participant RedeemQueue
+    participant TQOracle as TQ Oracle
+    participant Oracle
+    participant Vault
+    participant FeeManager
+    participant ShareManager
+
+    opt DEPOSIT FLOW
+    User->>DepositQueue: deposit(assets)
+    Note over DepositQueue: Request queued<br/>(time-delayed)
+
+    Note over TQOracle: Scheduled run...
+    TQOracle->>TQOracle: Aggregate balances<br/>(vault + subvaults + protocols)
+    TQOracle->>TQOracle: Price assets, compute TVL
+    TQOracle->>Oracle: submitReports(Report[])
+
+    Oracle->>Oracle: Validate deviation & cooldown
+    Oracle->>Vault: handleReport(asset, priceD18, timestamps)
+
+    Vault->>FeeManager: calculateFee(priceD18, totalShares)
+    FeeManager-->>Vault: feeShares
+    Vault->>ShareManager: mint(feeRecipient, feeShares)
+    Vault->>FeeManager: updateState(priceD18)
+
+    Vault->>DepositQueue: handleReport(priceD18, depositTimestamp)
+    DepositQueue->>DepositQueue: shares = assets × priceD18 / 1e18
+    Note over DepositQueue: Shares claimable
+
+    User->>DepositQueue: claim()
+    DepositQueue->>ShareManager: mint(user, shares)
+    ShareManager-->>User: Vault shares
+    end
+
+    opt REDEEM FLOW
+    User->>RedeemQueue: redeem(shares)
+    Note over RedeemQueue: Request queued<br/>(time-delayed)
+
+    Note over TQOracle: Scheduled run...
+    TQOracle->>Oracle: submitReports(Report[])
+    Oracle->>Vault: handleReport(asset, priceD18, timestamps)
+
+    Vault->>RedeemQueue: handleReport(priceD18, redeemTimestamp)
+    RedeemQueue->>RedeemQueue: assets = shares × 1e18 / priceD18
+    Note over RedeemQueue: Batch created
+
+    Vault->>Vault: handleBatches() [Curator]
+    Vault->>Vault: Pull liquidity from subvaults if needed
+    Vault->>RedeemQueue: Transfer assets for batch
+
+    User->>RedeemQueue: claim()
+    RedeemQueue->>ShareManager: burn(user, shares)
+    RedeemQueue-->>User: Assets
+    end
+```
+
+**Key components:**
+
+- **Vault**: Central coordinator that holds idle assets and delegates to subvaults.
+- **Subvaults**: Isolated contracts deploying assets to external protocols (EigenLayer, Symbiotic, etc.).
+- **ShareManager**: Controls share minting/burning with whitelist/blacklist permissions.
+- **FeeManager**: Calculates performance fees (on gains) and protocol fees (time-accrued).
+- **Queues**: Batch user deposit/redeem requests; settled atomically when oracle reports arrive.
+- **Oracle**: Validates price reports against deviation thresholds and enforces cooldown periods.
+
+**The oracle report lifecycle:**
+
+1. Users submit deposit/redeem requests to queues (time-delayed).
+2. TQ Oracle computes TVL by aggregating vault + subvault + external protocol balances.
+3. TQ Oracle submits `Report[]` via `Oracle.submitReports()` through a Safe multisig.
+4. Oracle contract validates price deviation and cooldown, then calls `Vault.handleReport()`.
+5. Vault mints fee shares, updates FeeManager state, and propagates price to queues.
+6. Queues convert pending deposits→shares and pending redemptions→assets at the reported price.
+7. Users claim their shares or redeemed assets.
+
+TQ Oracle is the off-chain price authority—it computes what the on-chain Oracle consumes. The on-chain Oracle enforces security invariants (max deviation, timeouts, suspicious-report flagging), but relies on TQ Oracle for accurate TVL calculation across all asset positions.
+
 ## Extension Points
 - **Asset adapters**: Implement `BaseAssetAdapter`, register in `ADAPTER_REGISTRY`, and wire via `subvault_adapters`.
 - **Price adapters**: Implement `BasePriceAdapter` and append to `PRICE_ADAPTERS` to join the pricing chain.

README.md

@@ -51,178 +51,77 @@ Run the CLI with a vault address and dry-run flag to preview reports without sub
 - Run with minimal CLI arguments - config file is auto-detected
 - See `tq-oracle.toml.example` for complete configuration template
 
-### With Environment Variables
-
-- Create `.env` file for environment-specific settings
-- Set RPC endpoints, subvault addresses, and other non-secret configuration
-- **Important**: Always use environment variables or CLI flags for secrets (private keys, API keys) - never store them in TOML files
-- Run with vault address and any additional CLI options
 
 ### Configuration Options
 
-All configuration options can be set via CLI arguments, environment variables, or TOML config file.
-
-| CLI Option | Environment Variable | TOML Key | Default | Description |
-|------------|---------------------|-----------|---------|-------------|
-| `vault_address` (argument) | `TQ_ORACLE_VAULT_ADDRESS` | `vault_address` | *required* | Vault contract address passed as positional argument |
-| `--config` `-c` | - | - | Auto-detect | Path to TOML configuration file |
-| `--network` `-n` | `TQ_ORACLE_NETWORK` | `network` | `"mainnet"` | Network to report on (`mainnet`, `sepolia`, `base`) |
-| `--block-number` | `TQ_ORACLE_BLOCK_NUMBER` | `block_number` | Latest block | Block number to snapshot vault state |
-| `--vault-rpc` | `TQ_ORACLE_VAULT_RPC` | `vault_rpc` | Network default | RPC endpoint for the selected vault network |
-| `--dry-run/--no-dry-run` | `TQ_ORACLE_DRY_RUN` | `dry_run` | `true` | Preview report without submitting a Safe transaction |
-| `--ignore-empty-vault/--require-nonempty-vault` | `TQ_ORACLE_IGNORE_EMPTY_VAULT` | `ignore_empty_vault` | `false` | Skip failure when vault holds zero assets |
-| `--ignore-timeout-check/--enforce-timeout-check` | `TQ_ORACLE_IGNORE_TIMEOUT_CHECK` | `ignore_timeout_check` | `false` | Skip minimum interval guard between reports |
-| `--ignore-active-proposal-check/--enforce-active-proposal-check` | `TQ_ORACLE_IGNORE_ACTIVE_PROPOSAL_CHECK` | `ignore_active_proposal_check` | `false` | Skip duplicate active proposal guard |
-| `--log-level` | `TQ_ORACLE_LOG_LEVEL` | `log_level` | `"INFO"` | Override logging verbosity (`TRACE`, `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`) |
-| `--show-config` | - | - | `false` | Dump effective configuration (with secrets redacted) and exit |
-
-#### TOML-Only Options (Not available via CLI)
-
-| TOML Key | Default | Description |
-|----------|---------|-------------|
-| `max_calls` | `3` | Maximum number of RPC retry attempts |
-| `rpc_max_concurrent_calls` | `5` | Maximum concurrent RPC connections |
-| `rpc_delay` | `0.15` | Delay between RPC calls (seconds) |
-| `rpc_jitter` | `0.10` | Random jitter for RPC delays (seconds) |
-| `price_warning_tolerance_percentage` | `0.5` | Price deviation warning threshold (%) |
-| `price_failure_tolerance_percentage` | `1.0` | Price deviation failure threshold (%) |
-
-### Advanced Configuration: Subvault Adapters
-
-TQ Oracle supports configuring multiple asset adapters per subvault address, allowing you to compose TVL from various sources.
-
-**Configuration via TOML:**
-
-- Specify target subvault address
-- List additional adapters to run for this subvault
-- Option to skip default idle balances check
-- Option to skip subvault existence validation
-
-**Available Asset Adapters:**
-
-- `idle_balances` - Checks for idle USDC balances not yet deployed
-
-See `tq-oracle.toml.example` for complete configuration examples.
+> [!IMPORTANT]  
+> See [SETTINGS.md](./SETTINGS.md) for complete guide.
 
 ### Usage Examples
 
 **Run with auto-detected config file:**
 
 ```bash
 # Loads from tq-oracle.toml or ~/.config/tq-oracle/config.toml
-tq-oracle
+uv run tq-oracle
 ```
 
 **Run with explicit vault address:**
 
 ```bash
 # Override vault address from config
-tq-oracle 0xYourVaultAddress
+uv run tq-oracle 0xYourVaultAddress
 ```
 
 **Run with custom config file:**
 
 ```bash
-tq-oracle --config path/to/custom-config.toml
+uv run tq-oracle --config path/to/custom-config.toml
 ```
 
 **Run with network override:**
 
 ```bash
-tq-oracle --network sepolia 0xYourVaultAddress
+uv run tq-oracle --network sepolia 0xYourVaultAddress
 ```
 
 **Preview configuration without running:**
 
 ```bash
-tq-oracle --show-config
+uv run tq-oracle --show-config
 ```
 
 **Increase verbosity for debugging:**
 
 ```bash
-tq-oracle --log-level DEBUG
+uv run tq-oracle --log-level DEBUG
 ```
 
-**Common Usage Patterns:**
-
-- **Dry-run on mainnet**: Preview report generation without submitting to chain
-- **Testnet execution**: Run against testnet with Safe multi-sig for testing
-- **Pre-deployment testing**: Test with empty vaults using ignore flags
-
 ## Architecture
 
 ```sh
 src/tq_oracle/
-├── main.py                           # CLI entry point (Typer)
-├── settings.py                       # Configuration management (pydantic-settings)
-├── state.py                          # Application state container
-├── pipeline/                         # Orchestration pipeline
-├── domain/                           # Core domain models
-├── adapters/                         # Protocol adapters
-├── processors/                       # Data processing utilities
-├── report/                           # Report generation and publishing
-├── safe/                             # Safe transaction building
-├── checks/                           # Pre-flight validation orchestration
-└── abis/                             # Contract ABIs (JSON)
+├── main.py              # CLI entry point (Typer)
+├── settings.py          # Configuration management (pydantic-settings)
+├── state.py             # Application state container
+├── abi.py               # ABI loading utilities
+├── constants.py         # Application constants
+├── logger.py            # Logging configuration
+├── pipeline/            # Orchestration (preflight → assets → pricing → report)
+├── adapters/            # Protocol adapters
+│   ├── asset_adapters/      # Fetch asset balances (e.g. idle, stakewise, streth)
+│   ├── price_adapters/      # Fetch asset prices (cow_swap, eth, pyth)
+│   ├── check_adapters/      # Pre-flight checks (active_submit, timeout)
+│   └── price_validators/    # Price validation
+├── processors/          # Data aggregation and TVL computation
+├── checks/              # Pre-flight validation orchestration
+├── report/              # Report encoding, generation, and publishing
+├── abis/                # Contract ABIs (JSON)
+└── tests/               # Mirrors src structure
 ```
 
-## Pre-Flight Checks
-
-Before processing TVL data, TQ Oracle runs automated pre-flight validation checks to ensure data integrity:
-
-- **Safe State Validation**: Ensures no duplicate or pending reports exist
-- **Check Retry Logic**: Automatically retries failed checks with exponential backoff when recommended
-
-These checks prevent race conditions and ensure accurate TVL snapshots by detecting ongoing cross-chain transfers that could affect asset balances. You can bypass individual guards when needed via the CLI flags `--ignore-empty-vault`, `--ignore-timeout-check/--enforce-timeout-check`, and `--ignore-active-proposal-check/--enforce-active-proposal-check`.
-
-## Adding New Adapters
-
-### Asset Adapters
-
-Asset adapters fetch asset holdings from specific protocols (e.g., Aave, Lido).
-
-Quick overview:
-
-1. **Create adapter file** in `src/tq_oracle/adapters/asset_adapters/` implementing `BaseAssetAdapter`
-2. **Register adapter** in `src/tq_oracle/adapters/asset_adapters/__init__.py`'s `ADAPTER_REGISTRY`
-3. **Write integration tests** in `tests/adapters/asset_adapters/`
-4. **Add asset addresses** to `src/tq_oracle/constants.py` if needed
-
-The adapter name in the registry is used in the `[[subvault_adapters]]` configuration's `additional_adapters` field.
-
-### Price Adapters
-
-Price adapters fetch USD prices for assets from price oracles (e.g., Pyth).
-
-1. **Create adapter file** in `src/tq_oracle/adapters/price_adapters/` implementing `BasePriceAdapter`
-2. **Register adapter** in `src/tq_oracle/adapters/price_adapters/__init__.py`'s `PRICE_ADAPTERS` list
-3. **Implement async `fetch_prices()` method** to query oracle and return price data
-4. **Write unit tests** in `tests/adapters/price_adapters/`
-
-### Price Validators
-
-Price validators cross-check prices from the main price adapters against reference sources to detect anomalies or manipulation. They run after price fetching and can issue warnings or halt execution if prices deviate beyond configured thresholds.
-
-1. **Create validator file** in `src/tq_oracle/adapters/price_validators/` implementing `BasePriceValidator`
-2. **Register validator** in `src/tq_oracle/adapters/price_validators/__init__.py`'s `PRICE_VALIDATORS` list
-3. **Implement async `validate_prices()` method** to cross-check prices and return validation results
-4. **Configure tolerance thresholds** in settings for warning and failure levels
-5. **Write unit tests** in `tests/adapters/price_validators/`
-
-Validators respect tolerance thresholds configured in `settings.py`:
-
-- `price_warning_tolerance_percentage` (default: 0.5%) - Issues warnings
-- `price_failure_tolerance_percentage` (default: 1.0%) - Halts execution
-
-## Development
-
-- **Install dependencies**: Use `uv sync --all-extras` for development dependencies
-- **Run tests**: Execute test suite with pytest
-- **Lint code**: Check code quality with ruff
-- **Format code**: Apply consistent formatting with ruff format
-
----
+> [!IMPORTANT]  
+> See [ARCHITECTURE.md](./ARCHITECTURE.md) for complete rundown.
 
 ## External Links