Token truncation handling

[jxnl]

feat: add truncation-aware retry handling with failfast and auto-ramp

Describe your changes

This enhancement introduces two new parameters to improve how Instructor handles model response truncation:

1. failfast_on_truncation=True: When enabled, Instructor will immediately raise an IncompleteOutputException if a model's response is truncated due to max_tokens limits. This prevents wasted validation retries when the issue is insufficient output length, not malformed JSON.
2. max_tokens_auto_ramp: This option allows Instructor to automatically increase the max_tokens (or equivalent model-specific token limits like max_output_tokens, maxTokens) for subsequent retries when truncation is detected. It's configurable with a multiplier, an optional cap, and max_attempts to control the ramping behavior. This addresses truncation by increasing the token budget only when needed, optimizing rate limits and concurrency.

These features are integrated into the instructor.patch function and the Instructor client's create methods, and are supported by updated documentation and new unit tests.

Issue ticket number and link

567-252, #1962

Checklist before requesting a review

• I have performed a self-review of my code
• If it is a core feature, I have added thorough tests.
• If it is a core feature, I have added documentation.

---

Linear Issue: 567-252

 

[cursor]

Cursor Agent can help with this pull request. Just @cursor in comments and I'll start working on changes in this branch.
_{Learn more about Cursor Agents}

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	instructor	c807efa	Commit Preview URL Branch Preview URL	Jan 16 2026, 02:41 PM