Skip to content

Add streaming support for xAI provider #1663

@jxnl

Description

@jxnl

Description

The xAI provider currently supports synchronous and asynchronous operations with both JSON and TOOLS modes, but does not yet support streaming responses (create_iterable and create_partial).

Background

xAI's SDK has a different streaming API compared to other providers. Instead of passing stream=True as a parameter, xAI uses a .stream() method on the chat object that returns tuples of (response, chunk).

Example from xAI docs:

for response, chunk in chat.stream():
    print(chunk.content, end="", flush=True)

What needs to be done

  1. Implement streaming support in instructor/client_xai.py for both sync and async clients
  2. Handle the conversion from xAI's streaming format to the format expected by instructor's streaming infrastructure
  3. Add comprehensive tests in tests/llm/test_xai/test_stream.py covering:
    • create_iterable for multiple object extraction
    • create_partial for partial object streaming
    • Both JSON and TOOLS modes
    • Both sync and async operations

Technical considerations

  • The implementation needs to yield response chunks in a format compatible with instructor's process_response function
  • For JSON mode: chunks should be extracted from the content
  • For TOOLS mode: chunks need to be extracted from tool calls
  • The streaming generator should yield chunks as they arrive, not accumulate all content first

References

  • xAI Streaming Documentation
  • Current xAI implementation: instructor/client_xai.py
  • Example streaming implementations: See OpenAI or Anthropic providers

This would be a great first contribution for someone familiar with async generators and streaming APIs!

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestgood first issueGood for newcomerspythonPull requests that update python codesize:MThis PR changes 30-99 lines, ignoring generated files.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions