-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Fix DeepSeek provider NameError and add streaming support #3033
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
harryslimes
wants to merge
9
commits into
letta-ai:main
Choose a base branch
from
harryslimes:fix/deepseek-provider-3032
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fixes letta-ai#3032 This commit addresses three critical issues with the DeepSeek provider: 1. Fixed NameError for undefined '_Message' type - Changed return type in map_messages_to_deepseek_format() from List[_Message] to List[ChatMessage] - Changed parameter type in build_deepseek_chat_completions_request() from List[_Message] to List[PydanticMessage] - The _Message type was never imported or defined in the module 2. Added streaming support for DeepSeek provider - Added ProviderType.deepseek to streaming interface conditions in simple_llm_stream_adapter.py - Added ProviderType.deepseek to streaming interface conditions in letta_llm_stream_adapter.py - Added ProviderType.deepseek to streaming interface conditions in letta_agent.py - DeepSeek uses OpenAI-compatible API and DeepseekClient already has stream_async() implemented 3. Fixed None handling in message processing - Added None checks in merge_tool_message() to prevent concatenating None values - Added None check in handle_assistant_message() for tool_calls iteration - Added content None guard in handle_assistant_message() to ensure content is never None - Added None handling in map_messages_to_deepseek_format() for all string concatenations - Fixes 'NoneType' object errors when processing messages with None content or tool_calls The DeepSeek provider now works correctly for both streaming and non-streaming requests with the deepseek-reasoner (R1) and other DeepSeek models.
Addresses reasoning display issues for issue letta-ai#3032 1. Fixed reasoning content handling in DeepseekClient - Keep reasoning_content in response instead of moving it to content field - Content field is set to None since it only contained tool call JSON - This allows the UI to properly display reasoning from deepseek-reasoner model 2. Added DeepSeek Reasoner to reasoning model detection - Added is_deepseek_reasoning_model() check in LLMConfig - Configured as native reasoning model (similar to OpenAI o1/o3) - Always enables reasoning for deepseek-reasoner (non-togglable) - Updated apply_reasoning_setting_to_config() to handle DeepSeek The deepseek-reasoner model now properly displays its native reasoning content in the Letta UI, similar to other reasoning models like Anthropic Claude and OpenAI o1/o3 series.
Process reasoning_content from ChatCompletionChunk delta for models that include it in streaming responses (like DeepSeek Reasoner). This enables real-time display of reasoning as it's generated. Changes: - Added reasoning_messages list to track ReasoningMessage chunks - Process message_delta.reasoning_content and emit ReasoningMessage - Updated get_content() to include accumulated ReasoningContent - Import ReasoningContent from letta_message_content This allows DeepSeek Reasoner's native reasoning to be displayed in real-time in the UI, similar to Anthropic Claude and other reasoning models.
Implements streaming JSON function call parsing for deepseek-reasoner model, which doesn't natively support tool calling but outputs function calls as JSON. Key changes: - Add JSON buffering and parsing in OpenAI streaming interface - Detects both single function calls ({...}) and parallel calls ([{...}]) - Buffers content chunks until complete JSON is received - Sets streaming interface state for agent to execute tool after streaming - Improve DeepSeek function calling instructions - More explicit JSON format requirements - Remove tools/tool_choice params to prevent fallback to deepseek-chat - Parse JSON function calls in non-streaming mode (deepseek_client.py) - Handles single function call objects with proper tool_call conversion - Clean up debug logging statements Limitations: - For parallel function calls, only the first call is executed (full parallel support requires broader streaming interface refactor) Tested with deepseek-reasoner (R1) model - streaming reasoning content and function calling both work correctly.
Adds unit and integration tests for DeepSeek Reasoner functionality: Unit tests (TestJSONFunctionCallParser): - Parse complete single function calls - Parse parallel function calls (arrays) - Handle incomplete JSON (continue buffering) - Detect non-function JSON - Handle malformed JSON - Parse nested JSON arguments - Handle string-encoded arguments Mock-based tests (TestDeepSeekStreamingWithMocks): - Stream reasoning content chunks correctly - Parse and convert single function calls to ToolCallMessage - Handle parallel function calls (use first call) - Verify streaming interface state is set for agent execution Integration test (optional, @pytest.mark.integration): - End-to-end test with real DeepSeek API - Verify reasoning content and function calling work together - Requires DEEPSEEK_API_KEY environment variable Tests follow existing Letta conventions from test_llm_clients.py and test_sonnet_nonnative_reasoning_buffering.py.
The _try_parse_json_function_call method was added to SimpleOpenAIStreamingInterface, not OpenAIStreamingInterface. Updated all test imports and fixtures to use the correct class. All 7 unit tests now pass: - Parse complete single function calls ✓ - Parse parallel function calls (arrays) ✓ - Handle incomplete JSON (continue buffering) ✓ - Detect non-function JSON ✓ - Handle malformed JSON ✓ - Parse nested JSON arguments ✓ - Handle string-encoded arguments ✓
Skip mock-based streaming tests and integration test by default: - Mock streaming tests require complex AsyncStream mocking (context manager protocol) - Integration test requires DEEPSEEK_API_KEY and running server - Core functionality is well-covered by 7 passing unit tests Test results: - 7 unit tests passing ✓ (JSON parser functionality) - 4 tests properly skipped (mock streaming + integration) The unit tests thoroughly cover the core JSON parsing logic which is the critical new functionality. Mock/integration tests can be added later if needed.
- Add TestInterfaceMethods with 9 tests for get_content() and get_tool_call_object() - Add TestNoneHandling with 6 tests for None handling bug fixes - Enhance all existing tool call tests with interface state verification - Fix imports to use correct modules (letta.schemas.message, letta.schemas.letta_message_content) - Remove obsolete Python 3.10 cache files from git tracking - All 31 tests passing Tests now cover: - JSON function call parsing (7 tests) - Interface method functionality (9 tests) - DeepSeek Chat streaming with native tool calling (4 tests) - DeepSeek Reasoner streaming with JSON parsing (5 tests) - None handling for reasoning_content, content, and tool_calls (6 tests)
- Update openai_streaming_interface.py with DeepSeek Reasoner support - Update openai_client.py for DeepSeek provider integration - Update mcp_tests configuration - Remove old test_deepseek_reasoner_streaming.py (replaced by test_deepseek_streaming.py)
rideam
pushed a commit
to ritza-co/letta-repo
that referenced
this pull request
Oct 14, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #3032
This commit addresses three critical issues with the DeepSeek provider:
Fixed NameError for undefined '_Message' type
Added streaming support for DeepSeek provider
Fixed None handling in message processing
The DeepSeek provider now works correctly for both streaming and non-streaming requests with the deepseek-reasoner (R1) and other DeepSeek models.
Please describe the purpose of this pull request.
To fix issues with DeepSeek Provider support
How to test
Set DEEPSEEK_API_KEY env var to valid deepseek API.
Test model on agent as either deepseek-chat or deepseek-reasoner
Have you tested this PR?
Yes, works with both deepseek-chat and deepseek-reasoner now
Related issues or PRs
3032