Skip to content

feat(core): new system prompt configurator for validation feedback. #469

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 3, 2025

Conversation

samchon
Copy link
Member

@samchon samchon commented Jul 3, 2025

This pull request introduces a new validation feedback mechanism for AI function calling, along with updates to system prompts and orchestration logic to enhance error handling and correction capabilities. The most important changes include the addition of a specialized validation prompt, integration of the validation prompt into function calling workflows, and documentation updates to clarify the foundational behavior of the AI agent.

Validation Feedback Mechanism:

  • packages/core/prompts/validate.md: Added detailed documentation and examples for the IValidation.IFailure structure, including guidelines for analyzing validation errors, providing correction feedback, and handling common error scenarios such as type mismatches, format violations, and union type failures.

System Prompt Updates:

  • packages/core/src/constants/AgenticaSystemPrompt.ts: Introduced a new VALIDATE system prompt to guide AI agents in correcting function calling parameter generation errors. This prompt includes detailed instructions and examples for handling validation failures.

Orchestration Logic Enhancements:

Documentation Improvements:

@samchon samchon requested review from sunrabbit123 and Copilot July 3, 2025 13:00
@samchon samchon self-assigned this Jul 3, 2025
@samchon samchon added this to WrtnLabs Jul 3, 2025
@samchon samchon added documentation Improvements or additions to documentation enhancement New feature or request labels Jul 3, 2025
@samchon samchon marked this pull request as ready for review July 3, 2025 13:00
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds a new validation feedback prompt to both the full and micro Agentica system prompts, integrates it into the function-calling orchestration logic, and provides detailed documentation for the IValidation.IFailure structure and usage.

  • Introduce validate prompt in IAgenticaSystemPrompt and IMicroAgenticaSystemPrompt interfaces
  • Integrate the new VALIDATE default prompt into the correct function in call.ts
  • Add comprehensive validate.md documentation for validation feedback scenarios

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file
File Description
packages/core/src/structures/IMicroAgenticaSystemPrompt.ts Add validate system prompt definition and docs
packages/core/src/structures/IAgenticaSystemPrompt.ts Add validate system prompt definition and docs
packages/core/src/orchestrate/call.ts Use validate prompt in error-correction workflow
packages/core/src/constants/AgenticaSystemPrompt.ts Define default VALIDATE prompt constant
packages/core/prompts/validate.md New markdown file with detailed validation feedback guide
Comments suppressed due to low confidence (6)

packages/core/src/structures/IMicroAgenticaSystemPrompt.ts:106

  • [nitpick] The @default tag for this validate prompt does not include a link to the validate.md file. Consider adding a URL reference for consistency with other prompts.
  validate?: () => string;

packages/core/src/structures/IAgenticaSystemPrompt.ts:173

  • [nitpick] Like other prompts, you should include a @default link pointing to the validate.md file to make it easy for maintainers to find the built-in prompt source.
  validate?: () => string;

packages/core/src/constants/AgenticaSystemPrompt.ts:15

  • [nitpick] Add a comment or URL indicating that this VALIDATE constant corresponds to packages/core/prompts/validate.md, mirroring how other prompts reference their source files.
  VALIDATE:

packages/core/prompts/validate.md:16

  • The code fence opening here uses four backticks but is closed with three, which can break formatting. Ensure matching backtick counts for the TypeScript example.
````typescript

packages/core/prompts/validate.md:323

  • There is an empty code fence here with no content inside. Consider removing the extra backticks to avoid rendering issues.
**packages/core/src/orchestrate/call.ts:454**
* [nitpick] This new integration point for the `validate` prompt should have accompanying unit or integration tests to verify that both the custom and default validation prompts produce the expected content.
    content: ctx.config?.systemPrompt?.validate?.()
</details>

Copy link

pkg-pr-new bot commented Jul 3, 2025

Open in StackBlitz

@agentica/benchmark

npm i https://pkg.pr.new/wrtnlabs/agentica/@agentica/benchmark@469

@agentica/chat

npm i https://pkg.pr.new/wrtnlabs/agentica/@agentica/chat@469

agentica

npm i https://pkg.pr.new/wrtnlabs/agentica@469

@agentica/core

npm i https://pkg.pr.new/wrtnlabs/agentica/@agentica/core@469

create-agentica

npm i https://pkg.pr.new/wrtnlabs/agentica/create-agentica@469

@agentica/rpc

npm i https://pkg.pr.new/wrtnlabs/agentica/@agentica/rpc@469

@agentica/vector-selector

npm i https://pkg.pr.new/wrtnlabs/agentica/@agentica/vector-selector@469

commit: 123b63d

@samchon samchon merged commit 0ed868e into main Jul 3, 2025
13 checks passed
@samchon samchon deleted the feat/validate-prompt branch July 3, 2025 13:10
@github-project-automation github-project-automation bot moved this to Done in WrtnLabs Jul 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

1 participant