Skip to content

test: add e2e test for Qwen3 tool call streaming truncation issue #66

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

devin-ai-integration[bot]
Copy link
Contributor

Test for Qwen3 Tool Call Streaming Truncation Issue

This PR adds an end-to-end test that reproduces a streaming truncation issue specifically occurring with Qwen3-235B's tool call responses in OpenRouter.

Issue Details

  • When using Qwen3-235B through OpenRouter, tool call arguments get truncated mid-JSON (e.g., {"query": "latest news", "c instead of the complete {"query": "latest news", "count": 5})
  • The sequence ends prematurely with a reasoning field containing /tool_call>
  • The test verifies that the SDK correctly handles this edge case by coercing invalid JSON to an empty object

Test Implementation

  • Creates mock response chunks that simulate the truncation issue
  • Verifies the streaming output contains the expected tool call delta with truncated arguments
  • Checks that a reasoning part with /tool_call> content is present
  • Confirms that the SDK coerces invalid JSON arguments to an empty object {}

Link to Devin run

https://app.devin.ai/sessions/1289ebbccea141e787ae452cc850ad00

Requested by: Louis ([email protected])

Copy link
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@louisgv louisgv closed this May 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant