Skip to content

Conversation

@gtarpenning
Copy link
Member

@gtarpenning gtarpenning commented Feb 2, 2026

Description

strict refactor of our largest server file. it is becoming impossible for agents to easily edit, I suspect there is similar fatigue among humans too.

New structure:

clickhouse_query_layer/
├── trace_server.py           # Main orchestration layer, composes all repositories
├── client.py                 # ClickHouse connection, pooling, _query/_insert/_command
├── batching.py               # Async batch management for call start/end inserts
├── schema.py                 # Pydantic models for CH insertables (calls, objects, files, etc.)
├── settings.py               # Configuration constants (batch sizes, limits, table suffixes)
├── migrator.py               # Schema migrations for replicated/distributed modes
├── calls.py                  # CallsRepository: CRUD + streaming + stats
├── objects.py                # ObjectsRepository: CRUD + versioning
├── tables.py                 # TablesRepository: row CRUD + queries
├── files.py                  # FilesRepository: chunked file storage
├── feedback.py               # FeedbackRepository: annotations/reactions
├── refs.py                   # RefsRepository: reference resolution
├── costs.py                  # CostsRepository: token cost lookups
├── annotation_queues.py      # AnnotationQueuesRepository: queue management
├── threads.py                # ThreadsRepository: conversation threads
├── stats.py                  # StatsRepository: project/call statistics
├── otel.py                   # OtelRepository: OpenTelemetry span export
├── v2_api.py                 # V2ApiRepository: newer API endpoints
├── completions.py            # CompletionsRepository: LLM completion proxying
├── query_builders/           # SQL query builders for ClickHouse
│   ├── annotation_queues.py  # Queue item queries, pop/add operations
│   ├── objects.py            # Object metadata queries, val queries
│   ├── project.py            # Project stats queries
│   ├── tables.py             # Table row queries with pagination
│   ├── threads.py            # Thread listing queries (CH + SQLite variants)
│   └── calls/                # Complex call query building
│       ├── calls_query_builder.py      # Main CallsQuery class with filters/sorting
│       ├── call_metrics_query_builder.py # Aggregation metrics (avg, sum, etc.)
│       ├── usage_query_builder.py      # Usage stats with time bucketing
│       ├── object_ref_query_builder.py # Object reference joins and lookups
│       ├── optimization_builder.py     # Query optimization passes
│       ├── stats_query_base.py         # Shared base for stats queries
│       ├── cte.py                      # CTE (Common Table Expression) helpers
│       └── utils.py                    # SQL formatting utilities

Testing

Existing coverage

@wandbot-3000
Copy link

wandbot-3000 bot commented Feb 2, 2026

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants