-
Notifications
You must be signed in to change notification settings - Fork 92
Tiho/agent skills integration #1270
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
5cf7df6
7aa816d
3346216
64ccc38
046c704
b3bab49
1538bdd
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| [submodule "agent-skills"] | ||
| path = agent-skills | ||
| url = https://github.com/mongodb/agent-skills.git |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,151 @@ | ||
| # Agent Skills Integration Tracker | ||
|
|
||
| This document tracks the implementation of MongoDB Agent Skills integration into the VS Code extension. | ||
| See the full integration plan in the [agent-skills repo](https://github.com/mongodb-js/agent-skills/blob/main/docs/VSCode%20Integration%20Plan.md). | ||
|
|
||
| --- | ||
|
|
||
| ## Repository Boundaries | ||
|
|
||
| ### `agent-skills` repo (content & validation) | ||
|
|
||
| | # | Task | Status | Notes | | ||
| |---|------|--------|-------| | ||
| | 1 | Author `mongodb-mcp-setup` skill | ✅ Done | Exists in `skills/mongodb-mcp-setup/`; frontmatter validated | | ||
| | 2 | Author `mongodb-query-generator` skill | ✅ Done | Exists in `skills/mongodb-query-generator/`; frontmatter validated | | ||
| | 3 | Author `mongodb-client-management` skill | ✅ Done | Connection pooling, driver config, error handling, lifecycle | | ||
| | 4 | Author `mongodb-data-modeling` skill | ✅ Done | Embedding vs referencing, schema patterns, anti-patterns | | ||
| | 5 | Author `mongodb-query-optimization` skill | ✅ Done | ESR rule, explain plans, index strategies, pipeline optimization | | ||
| | 6 | Author `mongodb-stream-processing` skill | ✅ Done | Change streams, resume tokens, event-driven patterns | | ||
| | 7 | Author `mongodb-search-ai` skill | ✅ Done | Atlas Search, vector search, RAG pipelines, hybrid search | | ||
| | 8 | Author `mongodb-transactions` skill | ✅ Done | Multi-doc transactions, read/write concerns, causal consistency | | ||
| | 9 | Skill validation CI workflow | ✅ Done | `.github/workflows/validate-skills.yml` + `.github/scripts/validate-skills.sh` | | ||
| | 10 | Publish as npm package (`@mongodb-js/agent-skills`) | ✅ Done | `package.json` created with name `@mongodb-js/agent-skills`, version `0.1.0`, `files: ["skills/"]` | | ||
| | 11 | Testing / evals harness | ✅ Done | `testing/mongodb-query-generator/` contains evals and workspace fixtures | | ||
|
|
||
| ### `vscode` repo — this repo (distribution & integration) | ||
|
|
||
| | # | Task | Status | Notes | | ||
| |---|------|--------|-------| | ||
| | 1 | Add `chatSkills` contribution point to `package.json` | ✅ Done | All 8 skills registered under `contributes.chatSkills` | | ||
| | 2 | Bump `engines.vscode` to `^1.100.0` | ✅ Done | Already `^1.101.1` — no change needed | | ||
| | 3 | Add `mongodb.agentSkills.enabled` setting | ✅ Done | Added under `contributes.configuration.properties` | | ||
| | 4 | Build pipeline to bundle skills from `agent-skills` repo | ✅ Done | `@mongodb-js/agent-skills` added as dependency; `.vscodeignore` exception ensures skills are included in VSIX | | ||
| | 5 | Verify MCP co-registration | ✅ Done | MCP server auto-starts via `MCPController`; tools registered via `vscode.lm.registerMcpServerDefinitionProvider('mongodb', ...)`. Skills reference `mcp__mongodb__*` which Copilot resolves through the registered MCP server. | | ||
| | 6 | Add telemetry for skill usage | ✅ Done | `AgentSkillInvokedTelemetryEvent` and `AgentSkillCompletedTelemetryEvent` added to `telemetryEvents.ts`; convenience methods `trackAgentSkillInvoked()` and `trackAgentSkillCompleted()` added to `TelemetryService` | | ||
| | 7 | End-to-end testing | ✅ Done | Skill registration, file existence, and frontmatter validation tests added to `extension.test.ts` | | ||
|
|
||
| --- | ||
|
|
||
| ## Implementation Details | ||
|
|
||
| ### 1. `chatSkills` Contribution Point | ||
|
|
||
| Add to `package.json` under `contributes`: | ||
|
|
||
| ```json | ||
| { | ||
| "contributes": { | ||
| "chatSkills": [ | ||
| { "path": "./skills/mongodb-mcp-setup/SKILL.md" }, | ||
| { "path": "./skills/mongodb-query-generator/SKILL.md" }, | ||
| { "path": "./skills/mongodb-client-management/SKILL.md" }, | ||
| { "path": "./skills/mongodb-data-modeling/SKILL.md" }, | ||
| { "path": "./skills/mongodb-query-optimization/SKILL.md" }, | ||
| { "path": "./skills/mongodb-stream-processing/SKILL.md" }, | ||
| { "path": "./skills/mongodb-search-ai/SKILL.md" }, | ||
| { "path": "./skills/mongodb-transactions/SKILL.md" } | ||
| ] | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ### 2. Version Constraint | ||
|
|
||
| ```json | ||
| { "engines": { "vscode": "^1.100.0" } } | ||
| ``` | ||
|
|
||
| ### 3. Optional Settings | ||
|
|
||
| ```json | ||
| { | ||
| "contributes": { | ||
| "configuration": { | ||
| "properties": { | ||
| "mongodb.agentSkills.enabled": { | ||
| "type": "boolean", | ||
| "default": true, | ||
| "description": "Enable MongoDB Agent Skills for GitHub Copilot." | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ### 4. Build Pipeline — Bundling Skills | ||
|
|
||
| The `@mongodb-js/agent-skills` package is added as a regular dependency. The `chatSkills` paths point directly to `./node_modules/@mongodb-js/agent-skills/skills/<name>/SKILL.md`. A `.vscodeignore` exception (`!node_modules/@mongodb-js/agent-skills/**`) ensures these files are included in the packaged VSIX. | ||
|
|
||
| No custom copy script or build step is needed. | ||
|
|
||
| ### 5. MCP Co-registration | ||
|
|
||
| The extension already bundles the MongoDB MCP Server. Requirements: | ||
| - The MCP server must be registered so `mcp__mongodb__*` tools are available to Copilot. | ||
| - Existing MCP configuration (connection string, Atlas credentials) must flow through. | ||
| - The `mongodb-mcp-setup` skill guides users through credential setup if not yet configured. | ||
|
|
||
| ### 6. Telemetry | ||
|
|
||
| Instrument the following metrics: | ||
|
|
||
| | Metric | Description | | ||
| |--------|-------------| | ||
| | Skill name | Which skill was invoked | | ||
| | Completion | Whether the skill ran to completion or was halted | | ||
| | Client | Identify as `vscode` | | ||
| | Extension installed | Whether the user has the extension | | ||
| | Skills version | Version of the bundled Agent Skills package | | ||
|
|
||
| Additionally: hook into MCP server tool invocation events, adding a `source: "agent-skill"` property to distinguish skill-originated calls from direct tool calls. | ||
|
|
||
| ### 7. What Is NOT Needed | ||
|
|
||
| - No `vscode.languages` providers — skills operate through Copilot chat, not language services. | ||
| - No completion providers — slash commands come automatically from `chatSkills`. | ||
| - No extension activation logic — skills are static `package.json` contributions. | ||
| - No new UI surfaces — skills live entirely within Copilot chat. | ||
|
|
||
| --- | ||
|
|
||
| ## Architecture | ||
|
|
||
| ``` | ||
| ┌─────────────────────────────────────────────────┐ | ||
| │ VS Code + Copilot │ | ||
| │ │ | ||
| │ User prompt ──► Copilot ──► Skill Discovery │ | ||
| │ │ (name + description) │ | ||
| │ ▼ │ | ||
| │ Skill Loading (SKILL.md body) │ | ||
| │ │ │ | ||
| │ ▼ │ | ||
| │ Agent follows skill instructions │ | ||
| │ │ │ | ||
| │ ▼ │ | ||
| │ Calls MCP tools (mcp__mongodb__*) │ | ||
| │ │ │ | ||
| │ ┌─────────────────┴──────────────────────┐ │ | ||
| │ │ MongoDB MCP Server (npx) │ │ | ||
| │ │ collection-schema, find, aggregate, │ │ | ||
| │ │ collection-indexes, list-databases, │ │ | ||
| │ │ Atlas Admin API tools │ │ | ||
| │ └─────────────────┬──────────────────────┘ │ | ||
| │ │ │ | ||
| └────────────────────┼──────────────────────────────┘ | ||
| ▼ | ||
| MongoDB Atlas / Local DB | ||
| ``` | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -58,7 +58,8 @@ | |
| "test-install": "bash ./scripts/test-vsix-install.sh", | ||
| "ai-accuracy-tests": "env TS_NODE_FILES=true mocha -r ts-node/register --grep=\"${MOCHA_GREP}\" --file ./src/test/ai-accuracy-tests/test-setup.ts ./src/test/ai-accuracy-tests/ai-accuracy-tests.ts", | ||
| "analyze-bundle": "webpack --mode production --analyze", | ||
| "vscode:prepublish": "pnpm run clean && pnpm run compile:constants && pnpm run compile:resources && webpack --mode production", | ||
| "sync-skills": "node ./scripts/sync-agent-skills.ts", | ||
| "vscode:prepublish": "pnpm run sync-skills && pnpm run clean && pnpm run compile:constants && pnpm run compile:resources && webpack --mode production", | ||
| "check": "pnpm run lint && pnpm run depcheck", | ||
| "depcheck": "depcheck", | ||
| "package": "cross-env NODE_OPTIONS='--require ./scripts/no-npm-list-fail.js' vsce package --githubBranch main", | ||
|
|
@@ -1388,6 +1389,11 @@ | |
| "type": "number", | ||
| "default": 120000, | ||
| "description": "Controls how often the MongoDB MCP server runs the export cleanup process to remove expired files (in milliseconds). Default is 2 minutes (120000ms)." | ||
| }, | ||
| "mongodb.agentSkills.enabled": { | ||
| "type": "boolean", | ||
| "default": true, | ||
| "description": "Enable MongoDB Agent Skills for GitHub Copilot." | ||
| } | ||
| } | ||
| }, | ||
|
|
@@ -1426,6 +1432,14 @@ | |
| "id": "mongodb", | ||
| "label": "MongoDB MCP Server Definition Provider" | ||
| } | ||
| ], | ||
| "chatSkills": [ | ||
| { | ||
| "path": "./agent-skills/skills/mongodb-mcp-setup/SKILL.md" | ||
| }, | ||
| { | ||
|
Comment on lines
+1436
to
+1440
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is it not allowed to contribute an entire folder or do we have to list them 1 by 1?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It seems its not allowed to contribute the entire folder, yes. I will do a double check on that though. |
||
| "path": "./agent-skills/skills/mongodb-query-generator/SKILL.md" | ||
| } | ||
| ] | ||
| }, | ||
| "dependencies": { | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,101 @@ | ||
| #! /usr/bin/env node | ||
|
|
||
| /** | ||
| * Scans agent-skills/skills/ (git submodule) for SKILL.md files | ||
| * and updates the `contributes.chatSkills` array in package.json automatically. | ||
| * | ||
| * Usage: | ||
| * node scripts/sync-agent-skills.ts # update package.json in place | ||
| * node scripts/sync-agent-skills.ts --check # exit 1 if package.json is out of date | ||
| */ | ||
|
|
||
| import fs from 'fs'; | ||
| import path from 'path'; | ||
| import { fileURLToPath } from 'url'; | ||
|
|
||
| const __dirname = path.dirname(fileURLToPath(import.meta.url)); | ||
| const ROOT_DIR = path.join(__dirname, '..'); | ||
| const PACKAGE_JSON_PATH = path.join(ROOT_DIR, 'package.json'); | ||
| const SKILLS_DIR = path.join( | ||
| ROOT_DIR, | ||
| 'agent-skills', | ||
| 'skills', | ||
| ); | ||
|
|
||
| const CHECK_MODE = process.argv.includes('--check'); | ||
|
|
||
| function discoverSkills(): string[] { | ||
| if (!fs.existsSync(SKILLS_DIR)) { | ||
| console.warn( | ||
| `⚠ Skills directory not found at ${SKILLS_DIR}. Is the agent-skills submodule initialised?`, | ||
| ); | ||
| return []; | ||
| } | ||
|
|
||
| return fs | ||
| .readdirSync(SKILLS_DIR, { withFileTypes: true }) | ||
| .filter((entry) => { | ||
| if (!entry.isDirectory()) return false; | ||
| const skillMd = path.join(SKILLS_DIR, entry.name, 'SKILL.md'); | ||
| return fs.existsSync(skillMd); | ||
| }) | ||
| .map((entry) => entry.name) | ||
| .sort(); | ||
| } | ||
|
|
||
| function buildChatSkills( | ||
| skillNames: string[], | ||
| ): Array<{ path: string }> { | ||
| return skillNames.map((name) => ({ | ||
| path: `./agent-skills/skills/${name}/SKILL.md`, | ||
| })); | ||
| } | ||
|
|
||
| (async () => { | ||
| const skillNames = discoverSkills(); | ||
|
|
||
| if (skillNames.length === 0) { | ||
| console.log('No skills found — chatSkills will be set to an empty array.'); | ||
| } else { | ||
| console.log(`Found ${skillNames.length} skill(s): ${skillNames.join(', ')}`); | ||
| } | ||
|
|
||
| const packageJsonRaw = fs.readFileSync(PACKAGE_JSON_PATH, 'utf8'); | ||
| const packageJson = JSON.parse(packageJsonRaw); | ||
|
|
||
| const newChatSkills = buildChatSkills(skillNames); | ||
| const currentChatSkills: unknown[] = | ||
| packageJson.contributes?.chatSkills ?? []; | ||
|
|
||
| const currentJson = JSON.stringify(currentChatSkills); | ||
| const newJson = JSON.stringify(newChatSkills); | ||
|
|
||
| if (currentJson === newJson) { | ||
| console.log('✔ chatSkills in package.json is already up to date.'); | ||
| process.exit(0); | ||
| } | ||
|
|
||
| if (CHECK_MODE) { | ||
| console.error( | ||
| '✖ chatSkills in package.json is out of date. Run `pnpm sync-skills` to fix.', | ||
| ); | ||
| process.exit(1); | ||
| } | ||
|
|
||
| // Update in place | ||
| packageJson.contributes.chatSkills = newChatSkills; | ||
|
|
||
| fs.writeFileSync( | ||
| PACKAGE_JSON_PATH, | ||
| JSON.stringify(packageJson, null, 2) + '\n', | ||
| 'utf8', | ||
| ); | ||
|
|
||
| console.log( | ||
| `✔ Updated chatSkills in package.json (${currentChatSkills.length} → ${newChatSkills.length} skills).`, | ||
| ); | ||
| })().catch((error) => { | ||
| console.error(`Failed to sync agent skills: ${error.message}`); | ||
| process.exit(1); | ||
| }); | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should not be committed.