feat: new /v1/responses #161
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ructure - Updated OpenAI dependency to version 1.99.2 in both `pyproject.toml` files for `nilai-api` and `nilai-common`. - Enhanced response model in `responses_model.py` by adding new fields and improving type definitions. - Refactored response handling in `responses.py` to include usage tracking for input and output tokens. - Adjusted import statements in `__init__.py` to streamline model access.
- Updated return types in `route_and_execute_tool_call` and `process_tool_calls` to use `FunctionCallOutput`. - Improved error handling and logging in tool execution. - Adjusted input handling in `handle_responses_tool_workflow` to support lists of `ResponseInputParam`. - Added new imports for `FunctionCallOutput` and related types in `nilai_common` models.
- Changed `ResponseFunctionToolCall` to `ResponseFunctionToolCallParam` in multiple functions for better type consistency. - Enhanced `handle_responses_tool_workflow` to utilize new input item types and improved handling of tool call results. - Updated imports in `__init__.py` and other files to reflect new model structures.
…e tests architecture - Introduced new test files for HTTP and OpenAI client interactions with the nilAI API. - Implemented tests for various scenarios including health checks, model retrieval, chat completions, and response generation. - Enhanced test coverage for rate limiting and code execution features. - Removed outdated test file for code execution, consolidating tests into more relevant suites.
- Changed EC2 instance type from g4dn.xlarge to g6.xlarge in the CI workflow. - Updated the docker-compose command to use the new GPT-20B configuration file. - Added a new docker-compose file for the GPT-20B GPU service, including environment settings and health checks. - Updated the CI model reference in the test configuration to use the new GPT-20B model.
- Added a dummy API key for BRAVEE2B in the CI environment setup. - Updated the EC2 image ID to a new version in the CI configuration.
…ity group, key name, and region
…e new GPT-20B model
…nce health check logging
…rvice status visibility
…t new naming convention
…es.sh to align with new naming convention
…thod and changing ToolsConfig to use a list for implemented tools
…ng non supported functions
…nd adjust large payload handling
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
Implements the OpenAI-compatible
/v1/responsesAPI, a more flexible alternative to Chat Completions with structured input, tool calling, web search, and streaming support.Key Features
New
/v1/responsesendpointAdvanced Tool Handling
Web Search Integration
web_searchparamMultimodal Support
Architecture
Split
private.pyinto modular endpoints:/v1/chat/completions → chat.py/v1/responses → responses.pyNew
responses_tool_router.pyfor tool workflowsModular API models in
nilai-commonTechnical Highlights