Seer AI analysis suggestions may not always be accurate

[dcramer]

Summary

When analyzing errors in the search_events agent, Seer AI suggested adding maxToolRoundtrips: 3 to the generateText call. However, this property doesn't exist in the AI SDK's generateText options, leading to TypeScript errors.

Problem

1. Seer AI provides analysis and suggestions that may not always be accurate or applicable
2. Our tools should not blindly trust Seer's suggestions without validation
3. We need better error messaging to indicate when Seer's suggestions might be incorrect

Context

From issue MCP-SERVER-ECD: https://sentry.sentry.io/issues/6761713004/

• Error: AI_NoObjectGeneratedError: No object generated: could not parse the response.
• Seer suggested: "Add maxToolRoundtrips: 3 to generateText call"
• Reality: This property doesn't exist in the AI SDK

Proposed Solution

1. Add disclaimers in the analyze_issue_with_seer tool output indicating that suggestions should be validated
2. Include context about the tool's limitations in the response
3. Consider adding validation for common incorrect suggestions

Related Code

• packages/mcp-server/src/tools/search-events/agent.ts:491-499
• packages/mcp-server/src/tools/analyze-issue-with-seer/handler.ts

Labels

• bug
• tool-improvement
• seer-ai