update docs

Author: enitrat

AGENTS.md

@@ -6,23 +6,24 @@ This file documents conventions and checklists for making changes that affect th
 
 When adding a new documentation source (e.g., a new docs site or SDK) make sure to complete all of the following steps:
 
-1. TypeScript ingestion (packages/ingester)
+1. TypeScript ingestion (ingesters)
 
-   - Create an ingester class extending `BaseIngester` or `MarkdownIngester` under `packages/ingester/src/ingesters/`.
-   - Register it in `packages/ingester/src/IngesterFactory.ts`.
+   - Create an ingester class extending `BaseIngester` or `MarkdownIngester` under `ingesters/src/ingesters/`.
+   - Register it in `ingesters/src/IngesterFactory.ts`.
    - Ensure chunks carry correct metadata: `uniqueId`, `contentHash`, `sourceLink`, and `source`.
-   - Run `pnpm generate-embeddings` (or `generate-embeddings:yes`) to populate/update the vector store.
+   - Run the embeddings generator to populate/update the vector store:
+     - Prefer: `bun run src/generateEmbeddings.ts` (or `bun run src/generateEmbeddings.ts -y`)
+     - If you have scripts wired: `bun run generate-embeddings` (or `generate-embeddings:yes`)
 
-2. Agents (TS)
+2. Agents (Python)
 
-   - Add the new enum value to `packages/agents/src/types/index.ts` under `DocumentSource`.
-   - Verify Postgres vector store accepts the new `source` and filters on it (`packages/agents/src/db/postgresVectorStore.ts`).
+   - Add the new enum value to `python/src/cairo_coder/core/types.py` under `DocumentSource`.
+   - Ensure filtering by `metadata->>'source'` works with the new value in `python/src/cairo_coder/dspy/document_retriever.py`.
+   - Update the query processor resource descriptions in `python/src/cairo_coder/dspy/query_processor.py` (`RESOURCE_DESCRIPTIONS`). The module validates that every `DocumentSource` has a description.
 
 3. Retrieval Pipeline (Python)
 
-   - Add the new enum value to `python/src/cairo_coder/core/types.py` under `DocumentSource`.
-   - Ensure filtering by `metadata->>'source'` works with the new value in `python/src/cairo_coder/dspy/document_retriever.py`.
-   - Update the query processor resource descriptions in `python/src/cairo_coder/dspy/query_processor.py` (`RESOURCE_DESCRIPTIONS`).
+   - No extra steps beyond the above; the retriever already supports filtering by `metadata->>'source'`.
 
 4. Optimized Program Files (Python) — required
 
@@ -34,7 +35,7 @@ When adding a new documentation source (e.g., a new docs site or SDK) make sure
 
    - Ensure the new source appears where appropriate (e.g., `/v1/agents` output and documentation tables):
      - `API_DOCUMENTATION.md`
-     - `packages/ingester/README.md`
+     - `ingesters/README.md`
      - Any user-facing lists of supported sources
 
 6. Quick Sanity Check

API_DOCUMENTATION.md

@@ -58,14 +58,25 @@ Lists every agent registered in Cairo Coder.
       "openzeppelin_docs",
       "corelib_docs",
       "scarb_docs",
-      "starknet_js"
+      "starknet_js",
+      "starknet_blog"
     ]
   },
   {
-    "id": "scarb-assistant",
-    "name": "Scarb Assistant",
-    "description": "Specialized assistant for Scarb build tool",
-    "sources": ["scarb_docs"]
+    "id": "starknet-agent",
+    "name": "Starknet Agent",
+    "description": "Assistant for the Starknet ecosystem (contracts, tools, docs).",
+    "sources": [
+      "cairo_book",
+      "starknet_docs",
+      "starknet_foundry",
+      "cairo_by_example",
+      "openzeppelin_docs",
+      "corelib_docs",
+      "scarb_docs",
+      "starknet_js",
+      "starknet_blog"
+    ]
   }
 ]
 ```
@@ -82,6 +93,7 @@ Lists every agent registered in Cairo Coder.
 | `corelib_docs`      | Cairo core library docs                    |
 | `scarb_docs`        | Scarb package manager documentation        |
 | `starknet_js`       | StarknetJS guides and SDK documentation    |
+| `starknet_blog`     | Starknet blog posts and announcements      |
 
 ## Chat Completions
 

ingesters/README.md

@@ -32,6 +32,7 @@ The ingester currently supports the following documentation sources:
 6. **Core Library Docs** (`corelib_docs`): Cairo core library documentation
 7. **Scarb Docs** (`scarb_docs`): Scarb package manager documentation
 8. **StarknetJS Guides** (`starknet_js`): StarknetJS guides and tutorials
+9. **Starknet Blog** (`starknet_blog`): Starknet blog posts and announcements
 
 ## Architecture
 
@@ -112,17 +113,16 @@ const chunks = splitter.splitMarkdownToChunks(markdown);
 
 ## Usage
 
-To use the ingester package, run the `generateEmbeddings.ts` script:
+To use the ingester package, run the embeddings generator script:
 
 ```bash
-# From the root of the package
-pnpm run generate-embeddings
-
-# From the root of the project
-turbo run generate-embeddings
+# Preferred (direct bun):
+bun run src/generateEmbeddings.ts
+# Non-interactive (yes to prompts):
+bun run src/generateEmbeddings.ts -y
 ```
 
-This will prompt you to select a documentation source to ingest. You can also select "Everything" to ingest all sources.
+This will prompt you to select a documentation source to ingest (or use `-y` for all). You can also select "Everything" to ingest all sources.
 
 ## Adding a New Documentation Source
 

python/README.md

@@ -82,7 +82,6 @@ docker compose up postgres backend --build
 ```bash
  curl -X POST "http://localhost:3001/v1/chat/completions" \
   -H "Content-Type: application/json" \
-  -H "x-api-key: YOUR_API_KEY" \
   -d '{
     "messages": [
       {