Skip to content

Conversation

harryslimes
Copy link

@harryslimes harryslimes commented Oct 11, 2025

Fixes #3032

This commit addresses three critical issues with the DeepSeek provider:

  1. Fixed NameError for undefined '_Message' type

    • Changed return type in map_messages_to_deepseek_format() from List[_Message] to List[ChatMessage]
    • Changed parameter type in build_deepseek_chat_completions_request() from List[_Message] to List[PydanticMessage]
    • The _Message type was never imported or defined in the module
  2. Added streaming support for DeepSeek provider

    • Added ProviderType.deepseek to streaming interface conditions in simple_llm_stream_adapter.py
    • Added ProviderType.deepseek to streaming interface conditions in letta_llm_stream_adapter.py
    • Added ProviderType.deepseek to streaming interface conditions in letta_agent.py
    • DeepSeek uses OpenAI-compatible API and DeepseekClient already has stream_async() implemented
  3. Fixed None handling in message processing

    • Added None checks in merge_tool_message() to prevent concatenating None values
    • Added None check in handle_assistant_message() for tool_calls iteration
    • Added content None guard in handle_assistant_message() to ensure content is never None
    • Added None handling in map_messages_to_deepseek_format() for all string concatenations
    • Fixes 'NoneType' object errors when processing messages with None content or tool_calls

The DeepSeek provider now works correctly for both streaming and non-streaming requests with the deepseek-reasoner (R1) and other DeepSeek models.

Please describe the purpose of this pull request.
To fix issues with DeepSeek Provider support

How to test
Set DEEPSEEK_API_KEY env var to valid deepseek API.
Test model on agent as either deepseek-chat or deepseek-reasoner

Have you tested this PR?
Yes, works with both deepseek-chat and deepseek-reasoner now

Related issues or PRs
3032

Fixes letta-ai#3032

This commit addresses three critical issues with the DeepSeek provider:

1. Fixed NameError for undefined '_Message' type
   - Changed return type in map_messages_to_deepseek_format() from List[_Message] to List[ChatMessage]
   - Changed parameter type in build_deepseek_chat_completions_request() from List[_Message] to List[PydanticMessage]
   - The _Message type was never imported or defined in the module

2. Added streaming support for DeepSeek provider
   - Added ProviderType.deepseek to streaming interface conditions in simple_llm_stream_adapter.py
   - Added ProviderType.deepseek to streaming interface conditions in letta_llm_stream_adapter.py
   - Added ProviderType.deepseek to streaming interface conditions in letta_agent.py
   - DeepSeek uses OpenAI-compatible API and DeepseekClient already has stream_async() implemented

3. Fixed None handling in message processing
   - Added None checks in merge_tool_message() to prevent concatenating None values
   - Added None check in handle_assistant_message() for tool_calls iteration
   - Added content None guard in handle_assistant_message() to ensure content is never None
   - Added None handling in map_messages_to_deepseek_format() for all string concatenations
   - Fixes 'NoneType' object errors when processing messages with None content or tool_calls

The DeepSeek provider now works correctly for both streaming and non-streaming requests
with the deepseek-reasoner (R1) and other DeepSeek models.
Addresses reasoning display issues for issue letta-ai#3032

1. Fixed reasoning content handling in DeepseekClient
   - Keep reasoning_content in response instead of moving it to content field
   - Content field is set to None since it only contained tool call JSON
   - This allows the UI to properly display reasoning from deepseek-reasoner model

2. Added DeepSeek Reasoner to reasoning model detection
   - Added is_deepseek_reasoning_model() check in LLMConfig
   - Configured as native reasoning model (similar to OpenAI o1/o3)
   - Always enables reasoning for deepseek-reasoner (non-togglable)
   - Updated apply_reasoning_setting_to_config() to handle DeepSeek

The deepseek-reasoner model now properly displays its native reasoning content
in the Letta UI, similar to other reasoning models like Anthropic Claude and
OpenAI o1/o3 series.
Process reasoning_content from ChatCompletionChunk delta for models that include
it in streaming responses (like DeepSeek Reasoner). This enables real-time
display of reasoning as it's generated.

Changes:
- Added reasoning_messages list to track ReasoningMessage chunks
- Process message_delta.reasoning_content and emit ReasoningMessage
- Updated get_content() to include accumulated ReasoningContent
- Import ReasoningContent from letta_message_content

This allows DeepSeek Reasoner's native reasoning to be displayed in real-time
in the UI, similar to Anthropic Claude and other reasoning models.
Implements streaming JSON function call parsing for deepseek-reasoner model,
which doesn't natively support tool calling but outputs function calls as JSON.

Key changes:
- Add JSON buffering and parsing in OpenAI streaming interface
  - Detects both single function calls ({...}) and parallel calls ([{...}])
  - Buffers content chunks until complete JSON is received
  - Sets streaming interface state for agent to execute tool after streaming
- Improve DeepSeek function calling instructions
  - More explicit JSON format requirements
  - Remove tools/tool_choice params to prevent fallback to deepseek-chat
- Parse JSON function calls in non-streaming mode (deepseek_client.py)
  - Handles single function call objects with proper tool_call conversion
- Clean up debug logging statements

Limitations:
- For parallel function calls, only the first call is executed
  (full parallel support requires broader streaming interface refactor)

Tested with deepseek-reasoner (R1) model - streaming reasoning content
and function calling both work correctly.
Adds unit and integration tests for DeepSeek Reasoner functionality:

Unit tests (TestJSONFunctionCallParser):
- Parse complete single function calls
- Parse parallel function calls (arrays)
- Handle incomplete JSON (continue buffering)
- Detect non-function JSON
- Handle malformed JSON
- Parse nested JSON arguments
- Handle string-encoded arguments

Mock-based tests (TestDeepSeekStreamingWithMocks):
- Stream reasoning content chunks correctly
- Parse and convert single function calls to ToolCallMessage
- Handle parallel function calls (use first call)
- Verify streaming interface state is set for agent execution

Integration test (optional, @pytest.mark.integration):
- End-to-end test with real DeepSeek API
- Verify reasoning content and function calling work together
- Requires DEEPSEEK_API_KEY environment variable

Tests follow existing Letta conventions from test_llm_clients.py and
test_sonnet_nonnative_reasoning_buffering.py.
The _try_parse_json_function_call method was added to SimpleOpenAIStreamingInterface,
not OpenAIStreamingInterface. Updated all test imports and fixtures to use the
correct class.

All 7 unit tests now pass:
- Parse complete single function calls ✓
- Parse parallel function calls (arrays) ✓
- Handle incomplete JSON (continue buffering) ✓
- Detect non-function JSON ✓
- Handle malformed JSON ✓
- Parse nested JSON arguments ✓
- Handle string-encoded arguments ✓
Skip mock-based streaming tests and integration test by default:
- Mock streaming tests require complex AsyncStream mocking (context manager protocol)
- Integration test requires DEEPSEEK_API_KEY and running server
- Core functionality is well-covered by 7 passing unit tests

Test results:
- 7 unit tests passing ✓ (JSON parser functionality)
- 4 tests properly skipped (mock streaming + integration)

The unit tests thoroughly cover the core JSON parsing logic which is the
critical new functionality. Mock/integration tests can be added later if needed.
- Add TestInterfaceMethods with 9 tests for get_content() and get_tool_call_object()
- Add TestNoneHandling with 6 tests for None handling bug fixes
- Enhance all existing tool call tests with interface state verification
- Fix imports to use correct modules (letta.schemas.message, letta.schemas.letta_message_content)
- Remove obsolete Python 3.10 cache files from git tracking
- All 31 tests passing

Tests now cover:
- JSON function call parsing (7 tests)
- Interface method functionality (9 tests)
- DeepSeek Chat streaming with native tool calling (4 tests)
- DeepSeek Reasoner streaming with JSON parsing (5 tests)
- None handling for reasoning_content, content, and tool_calls (6 tests)
- Update openai_streaming_interface.py with DeepSeek Reasoner support
- Update openai_client.py for DeepSeek provider integration
- Update mcp_tests configuration
- Remove old test_deepseek_reasoner_streaming.py (replaced by test_deepseek_streaming.py)
rideam pushed a commit to ritza-co/letta-repo that referenced this pull request Oct 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Deepseek support not working

1 participant