feat: ✨ Added config file support #29

timbrinded · 2025-10-21T15:08:43Z

Configuration System Overhaul

This PR introduces comprehensive TOML-based configuration support alongside CLI arguments and environment variables, with clear precedence: CLI → ENV → TOML. The configuration loader (config_loader.py) handles auto-detection from standard paths (./tq-oracle.toml, ~/.config/tq-oracle/config.toml) and validates all settings with helpful error messages.

Key configuration enhancements:

TOML support for persistent configuration (secrets still require ENV/CLI)
Expanded RPC settings (max calls, concurrency, delays, jitter)
New pre-check retry/timeout controls
Chain-specific RPC endpoints (L1 and Hyperliquid)
Enhanced README with configuration precedence tables and examples

Asset Adapter Refactoring

Asset adapters now support multi-chain operations through a unified registry system (ADAPTER_REGISTRY). The AdapterChain enum distinguishes between L1 and Hyperliquid operations, allowing adapters to operate on different RPC endpoints.

IdleBalancesAdapter gains chain awareness with separate fetch_assets() (single subvault) and fetch_all_assets() (all subvaults) methods, supporting both L1 and Hyperliquid chains through constructor parameterization.

HyperliquidAdapter switches from TWAP to latest portfolio value with staleness validation (rejects values older than HL_MAX_PORTFOLIO_STALENESS_SECONDS). Improved error handling for empty/invalid history and better logging of portfolio age.

Orchestrator & Main

Orchestrator refactored for cleaner adapter composition with chain-aware asset collection. Configuration validation moved earlier in the pipeline with clearer error messages. Main entry point simplified with TOML config integration and improved help text.

…/config-file

* feat: add validation for asset prices in total_assets calculation (#35) * feat: add validation for asset prices in total_assets calculation - Implemented checks for invalid prices (<= 0) in the calculate_total_assets function. - Added error handling to raise ValueError with details of invalid prices. - Introduced a new test to ensure that invalid prices trigger the appropriate exception. * refactor: turn invalid prices computation into list comprehension * feat: add price validation for price adapter (#36) * fix: update WstETHAdapter and tests to set ETH base asset price to 1 - Changed the base asset price for ETH in WstETHAdapter from 0 to 1 to ensure accurate pricing. - Updated related tests to verify that the ETH price is now correctly set to 1 instead of 0. - Adjusted documentation to clarify the base asset pricing behavior. * feat: add price validation to BasePriceAdapter * refactor: change validate_prices method to synchronous in price adapters - Updated the validate_prices method in BasePriceAdapter to be synchronous instead of asynchronous. - Adjusted calls to validate_prices in ChainlinkAdapter, CowSwapAdapter, and WstETHAdapter accordingly. * feat: add TypedDict for network addresses in constants.py * feat: add NetworkAddresses instances * refactor: rename NetworkAddresses to NetworkTokens and update address constants * feat: add network configuration and asset retrieval to OracleCLIConfig * refactor: update WstETHAdapter to use dynamic asset addresses from config * refactor: enhance WstETHAdapter to enforce required ETH address and simplify checks * refactor: update CowSwapAdapter to use dynamic asset addresses and improve network configuration * refactor: update ChainlinkAdapter to use dynamic asset addresses from config and enforce required ETH address * refactor: update IdleBalancesAdapter to use dynamic asset addresses from config and enforce required ETH and USDC addresses * refactor: remove optional asset address attributes from Chainlink, CowSwap, and WstETH adapters * refactor: enforce required USDC address in HyperliquidAdapter and update asset address retrieval from config * refactor: update Chainlink test suite to use dynamic asset addresses from config * refactor: update WstETHAdapter tests to utilize dynamic asset addresses from config * refactor: enhance HyperliquidAdapter tests to utilize dynamic USDC addresses from config * refactor: update tests to utilize dynamic asset addresses from config for improved consistency * refactor: update Chainlink and CowSwap tests to utilize dynamic asset addresses from config for improved consistency * refactor: update Chainlink tests to include dynamic ETH address in price fetching for USDT and USDS on testnet * refactor: update import paths from config to settings in CowSwap and Hyperliquid test files

…/config-file

Copilot

Pull Request Overview

Copilot reviewed 48 out of 50 changed files in this pull request and generated 6 comments.

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

src/tq_oracle/processors/oracle_helper.py

src/tq_oracle/constants.py

tests/adapters/asset_adapters/test_idle_balances.py

src/tq_oracle/main.py

src/tq_oracle/settings.py

README.md

Co-authored-by: Copilot <[email protected]>

* docs * docs

Co-authored-by: Copilot <[email protected]>

Copilot

Pull Request Overview

Copilot reviewed 49 out of 51 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

src/tq_oracle/processors/oracle_helper.py:1

Inconsistent comment style: Line 89 is part of the docstring but uses # instead of proper docstring continuation. Move the Solidity sorting explanation into the main docstring or keep it as a standalone comment after the docstring closes.

from __future__ import annotations

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-10-24T07:55:46Z

src/tq_oracle/constants.py

+TOKEN_DECIMALS: dict[str, int] = {
+    addr: decimals
+    for addr, decimals in [
+        (ETH_MAINNET_ASSETS.get("USDC"), 6),
+        (SEPOLIA_ASSETS.get("USDC"), 6),
+        (ETH_MAINNET_ASSETS.get("USDT"), 6),
+        (ETH_MAINNET_ASSETS.get("USDS"), 18),
+    ]
+    if addr is not None
 }



The TOKEN_DECIMALS dictionary is constructed from hardcoded tuples that could become inconsistent with NetworkAssets definitions. Consider deriving decimals from a centralized constant or adding type hints to NetworkAssets to include decimals information.

Suggested change

TOKEN_DECIMALS: dict[str, int] = {

addr: decimals

for addr, decimals in [

(ETH_MAINNET_ASSETS.get("USDC"), 6),

(SEPOLIA_ASSETS.get("USDC"), 6),

(ETH_MAINNET_ASSETS.get("USDT"), 6),

(ETH_MAINNET_ASSETS.get("USDS"), 18),

]

if addr is not None

}

# Centralized decimals per asset symbol

ASSET_DECIMALS: dict[str, int] = {

"USDC": 6,

"USDT": 6,

"USDS": 18,

"ETH": 18,

"WETH": 18,

"WSTETH": 18,

}

# Collect all asset addresses from all NetworkAssets dicts

TOKEN_DECIMALS: dict[str, int] = {}

for assets in [ETH_MAINNET_ASSETS, SEPOLIA_ASSETS, BASE_ASSETS]:

for symbol, addr in assets.items():

if addr is not None and symbol in ASSET_DECIMALS:

TOKEN_DECIMALS[addr] = ASSET_DECIMALS[symbol]

README.md

Co-authored-by: Copilot <[email protected]>

src/tq_oracle/processors/oracle_helper.py

src/tq_oracle/pipeline/run.py

feat: ✨ Added config file support

a633db9

Copilot AI review requested due to automatic review settings October 21, 2025 15:08

This comment was marked as outdated.

Sign in to view

timbrinded added 2 commits October 21, 2025 20:14

tidy

273267e

tidy

26fcc8d

Copilot AI review requested due to automatic review settings October 21, 2025 20:05

This comment was marked as outdated.

Sign in to view

Merge branch 'master' of github.com:Moonsong-Labs/tq-oracle into feat…

2fa2735

…/config-file

timbrinded linked an issue Oct 22, 2025 that may be closed by this pull request

Map sub-vaults to asset adapters #26

Closed

timbrinded added 2 commits October 22, 2025 09:57

Merge branch 'master' of github.com:Moonsong-Labs/tq-oracle into feat…

a759f31

…/config-file

lint

5e26dc1

Copilot AI review requested due to automatic review settings October 22, 2025 08:58

This comment was marked as outdated.

Sign in to view

timbrinded added 2 commits October 22, 2025 14:04

refactor: ♻️ tidy

80f27fb

ci: 💚 fix ci

4afcd77

Copilot AI review requested due to automatic review settings October 22, 2025 13:44

This comment was marked as outdated.

Sign in to view

timbrinded added 2 commits October 22, 2025 15:31

fix: 🐛 fix asset order

02ca05d

fix: 🐛 fix oraclehelper issue

0c4620e

Copilot AI review requested due to automatic review settings October 22, 2025 14:59

This comment was marked as outdated.

Sign in to view

timbrinded added 3 commits October 22, 2025 16:09

fix

01b0c4f

refactor: ♻️ tidy

6c67985

Merge branch 'master' of github.com:Moonsong-Labs/tq-oracle into feat…

fd5649b

…/config-file

Copilot AI review requested due to automatic review settings October 22, 2025 19:25

This comment was marked as outdated.

Sign in to view

fmt

1e29902

timbrinded marked this pull request as ready for review October 22, 2025 19:39

Copilot AI review requested due to automatic review settings October 22, 2025 19:39

This comment was marked as outdated.

Sign in to view

matias-gonz and others added 2 commits October 23, 2025 14:55

test fix

02dab8c

Copilot AI review requested due to automatic review settings October 23, 2025 14:34

This comment was marked as outdated.

Sign in to view

timbrinded added 2 commits October 23, 2025 16:14

refactor: ♻️ pr comments

c230868

Merge branch 'master' of github.com:Moonsong-Labs/tq-oracle into feat…

69bcfc0

…/config-file

Copilot AI review requested due to automatic review settings October 23, 2025 15:20

This comment was marked as outdated.

Sign in to view

pr comments

d4195a7

timbrinded marked this pull request as ready for review October 23, 2025 15:45

timbrinded requested review from cheelahim, Copilot and matias-gonz October 23, 2025 15:45

Copilot AI reviewed Oct 23, 2025

View reviewed changes

Update constants.py

a96207b

Co-authored-by: Copilot <[email protected]>

Copilot AI review requested due to automatic review settings October 23, 2025 15:58

This comment was marked as outdated.

Sign in to view

timbrinded and others added 2 commits October 23, 2025 20:46

Feat/improved docs checks (#40)

aaebcde

* docs * docs

Update src/tq_oracle/adapters/asset_adapters/idle_balances.py

4fb5e55

Co-authored-by: Copilot <[email protected]>

Copilot AI review requested due to automatic review settings October 24, 2025 07:54

Update tq-oracle-example.toml

159f70a

Co-authored-by: Copilot <[email protected]>

Copilot AI reviewed Oct 24, 2025

View reviewed changes

Update README.md

a797678

Co-authored-by: Copilot <[email protected]>

Copilot AI review requested due to automatic review settings October 24, 2025 07:59

This comment was marked as outdated.

Sign in to view

cheelahim reviewed Oct 24, 2025

View reviewed changes

src/tq_oracle/processors/oracle_helper.py Show resolved Hide resolved

cheelahim reviewed Oct 24, 2025

View reviewed changes

src/tq_oracle/pipeline/run.py Show resolved Hide resolved

timbrinded merged commit 23a0044 into master Oct 24, 2025
6 checks passed

timbrinded deleted the feat/config-file branch October 24, 2025 11:07

-TOKEN_DECIMALS: dict[str, int] = {
-    addr: decimals
-    for addr, decimals in [
-        (ETH_MAINNET_ASSETS.get("USDC"), 6),
-        (SEPOLIA_ASSETS.get("USDC"), 6),
-        (ETH_MAINNET_ASSETS.get("USDT"), 6),
-        (ETH_MAINNET_ASSETS.get("USDS"), 18),
-    ]
-    if addr is not None
-}
+# Centralized decimals per asset symbol
+ASSET_DECIMALS: dict[str, int] = {
+    "USDC": 6,
+    "USDT": 6,
+    "USDS": 18,
+    "ETH": 18,
+    "WETH": 18,
+    "WSTETH": 18,
+}
+# Collect all asset addresses from all NetworkAssets dicts
+TOKEN_DECIMALS: dict[str, int] = {}
+for assets in [ETH_MAINNET_ASSETS, SEPOLIA_ASSETS, BASE_ASSETS]:
+    for symbol, addr in assets.items():
+        if addr is not None and symbol in ASSET_DECIMALS:
+            TOKEN_DECIMALS[addr] = ASSET_DECIMALS[symbol]

feat: ✨ Added config file support #29

feat: ✨ Added config file support #29

Uh oh!

Conversation

timbrinded commented Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Configuration System Overhaul

Asset Adapter Refactoring

Orchestrator & Main

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Copilot AI Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

timbrinded commented Oct 21, 2025 •

edited

Loading