-
-
Notifications
You must be signed in to change notification settings - Fork 827
Open
Labels
enhancementNew feature or requestNew feature or requestgood first issueGood for newcomersGood for newcomerspythonPull requests that update python codePull requests that update python codesize:MThis PR changes 30-99 lines, ignoring generated files.This PR changes 30-99 lines, ignoring generated files.
Description
Description
The xAI provider currently supports synchronous and asynchronous operations with both JSON and TOOLS modes, but does not yet support streaming responses (create_iterable
and create_partial
).
Background
xAI's SDK has a different streaming API compared to other providers. Instead of passing stream=True
as a parameter, xAI uses a .stream()
method on the chat object that returns tuples of (response, chunk)
.
Example from xAI docs:
for response, chunk in chat.stream():
print(chunk.content, end="", flush=True)
What needs to be done
- Implement streaming support in
instructor/client_xai.py
for both sync and async clients - Handle the conversion from xAI's streaming format to the format expected by instructor's streaming infrastructure
- Add comprehensive tests in
tests/llm/test_xai/test_stream.py
covering:create_iterable
for multiple object extractioncreate_partial
for partial object streaming- Both JSON and TOOLS modes
- Both sync and async operations
Technical considerations
- The implementation needs to yield response chunks in a format compatible with instructor's
process_response
function - For JSON mode: chunks should be extracted from the content
- For TOOLS mode: chunks need to be extracted from tool calls
- The streaming generator should yield chunks as they arrive, not accumulate all content first
References
- xAI Streaming Documentation
- Current xAI implementation:
instructor/client_xai.py
- Example streaming implementations: See OpenAI or Anthropic providers
This would be a great first contribution for someone familiar with async generators and streaming APIs!
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or requestgood first issueGood for newcomersGood for newcomerspythonPull requests that update python codePull requests that update python codesize:MThis PR changes 30-99 lines, ignoring generated files.This PR changes 30-99 lines, ignoring generated files.