feat(datatets-supervisor): improvement of the supervisor graph + langsmith datasets

[0xhijo]

Supervisor

• Update the prompt based on the cursor CLI prompt and claude code
• Creation of the handoff system to make the supervisor the possibility to call agent with/o query
• Add human-in-the-loop(HITL) in the different sub-agent of the graph
• Fix supervisor build-in directly from the langchain

Datatets

• Implementing of datasets
• Run datasets directly with csv file on different agent and node
• Creation of an AI judgement that will send the notes of the differetens run of the datasets

Core

• Add the thread_id for every turn of the agents to create a conversation system

Summary by CodeRabbit

Release Notes

• New Features

  • Added thread-based conversation tracking for persistent message and memory context across sessions.
  • Introduced agent selector helper for intelligent routing to specialized agents.
  • Enabled dataset evaluation capabilities for testing agent responses.
  • Added transfer mechanism for seamless agent-to-agent handoffs.

• Improvements

  • Enhanced request validation to reject empty or whitespace-only input.
  • Simplified memory retrieval and storage with thread-scoped context.
  • Streamlined WebSocket gateway for improved stability.

• Removals

  • Removed custom prompts configuration requirement; system now uses optimized defaults.

[coderabbitai]

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

📥 Commits

Reviewing files that changed from the base of the PR and between ec30c55 and 9bef418.

⛔ Files ignored due to path filters (6)

• datasets/supervisor-agentConfigurationHelper.dataset.csv is excluded by !**/*.csv
• datasets/supervisor-agentSelectorHelper.dataset.csv is excluded by !**/*.csv
• datasets/supervisor-mcpConfigurationHelper.dataset.csv is excluded by !**/*.csv
• datasets/supervisor-snakRagAgentHelper.dataset.csv is excluded by !**/*.csv
• datasets/supervisor-supervisor.dataset.csv is excluded by !**/*.csv
• pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml

📒 Files selected for processing (61)

• config/guards/default.guards.json (0 hunks)
• datasets/.gitignore (1 hunks)
• docker-compose.dev.yml (1 hunks)
• package.json (6 hunks)
• packages/agent/src/agents/core/snakAgent.ts (5 hunks)
• packages/agent/src/agents/core/supervisorAgent.ts (9 hunks)
• packages/agent/src/agents/graphs/constants/execution-constants.ts (1 hunks)
• packages/agent/src/agents/graphs/core-graph/agent.graph.ts (0 hunks)
• packages/agent/src/agents/graphs/core-graph/supervisor.graph.ts (4 hunks)
• packages/agent/src/agents/graphs/manager/memory/memory-db-manager.ts (11 hunks)
• packages/agent/src/agents/graphs/parser/memory/stm-parser.ts (0 hunks)
• packages/agent/src/agents/graphs/sub-graph/task-memory.graph.ts (9 hunks)
• packages/agent/src/agents/graphs/tools/memory.tool.ts (3 hunks)
• packages/agent/src/agents/langsmith/README.md (1 hunks)
• packages/agent/src/agents/langsmith/datasets.ts (1 hunks)
• packages/agent/src/agents/langsmith/run-datasets.ts (1 hunks)
• packages/agent/src/agents/operators/__tests__/agentSelector.spec.ts (0 hunks)
• packages/agent/src/agents/operators/supervisor/supervisorTools.ts (4 hunks)
• packages/agent/src/agents/operators/supervisor/tools/createAgentTool.ts (1 hunks)
• packages/agent/src/agents/operators/supervisor/tools/executeHandoffTools.ts (1 hunks)
• packages/agent/src/agents/operators/supervisor/tools/messageAskUserTools.ts (0 hunks)
• packages/agent/src/agents/operators/supervisor/tools/schemas/common.schemas.ts (1 hunks)
• packages/agent/src/agents/operators/supervisor/tools/schemas/message_ask_user.schema.ts (1 hunks)
• packages/agent/src/agents/operators/supervisor/tools/schemas/transfer_to_supervisorTools.ts (1 hunks)
• packages/agent/src/agents/operators/supervisor/tools/schemas/updateAgent.schema.ts (1 hunks)
• packages/agent/src/agents/operators/supervisor/utils/normalizeAgentValues.ts (2 hunks)
• packages/agent/src/index.ts (1 hunks)
• packages/agent/src/services/mcp/src/mcp.service.ts (1 hunks)
• packages/agent/src/shared/prompts/agents/agentSelector.prompt.ts (1 hunks)
• packages/agent/src/shared/prompts/agents/supervisor/specialist/agentConfigurationHelper.prompt.ts (1 hunks)
• packages/agent/src/shared/prompts/agents/supervisor/supervisor.prompt.ts (1 hunks)
• packages/agent/src/shared/types/graph.type.ts (1 hunks)
• packages/agent/src/shared/types/memory.type.ts (5 hunks)
• packages/agent/src/shared/types/streaming.type.ts (1 hunks)
• packages/agent/src/utils/agent-initialization.utils.ts (1 hunks)
• packages/core/src/common/agent/interfaces/agent.interface.ts (0 hunks)
• packages/core/src/common/constant/agents.constants.ts (1 hunks)
• packages/core/src/common/server/dto/agent/config.dto.ts (1 hunks)
• packages/core/src/common/server/dto/agent/message.dto.ts (4 hunks)
• packages/core/src/config/guards/guardsSchema.ts (0 hunks)
• packages/core/src/index.ts (0 hunks)
• packages/core/src/services/agent-validation.service.ts (2 hunks)
• packages/database/functions/add_agent_with_json.sql (0 hunks)
• packages/database/initdb/03-agents.sql (9 hunks)
• packages/database/initdb/04-messages.sql (3 hunks)
• packages/database/initdb/05a-memory.sql (13 hunks)
• packages/database/initdb/05b-memory.sql (33 hunks)
• packages/database/src/queries/agents/queries.ts (6 hunks)
• packages/database/src/queries/memory/queries.ts (10 hunks)
• packages/database/src/queries/message/queries.ts (1 hunks)
• packages/database/src/queries/rag/queries.ts (1 hunks)
• packages/database/src/queries/redis/queries.ts (1 hunks)
• packages/server/common/interceptors/error-logging.interceptor.ts (1 hunks)
• packages/server/src/agents.storage.ts (2 hunks)
• packages/server/src/controllers/agents.controller.ts (1 hunks)
• packages/server/src/controllers/gateway.controller.ts (2 hunks)
• packages/server/src/services/agent.service.ts (4 hunks)
• packages/server/src/utils/error-handler.ts (1 hunks)
• packages/workers/src/workers/agent-cfg-outbox.worker.ts (0 hunks)
• patches/@[email protected] (1 hunks)
• pnpm-workspace.yaml (0 hunks)

 _____________________________________________________________________________________________________________
< Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away. >
 -------------------------------------------------------------------------------------------------------------
  \
   \   (\__/)
       (•ㅅ•)
       / 　 づ

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch feat/langsmith-datasets

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}