Releases: BerriAI/litellm
Releases · BerriAI/litellm
v1.67.5-nightly
What's Changed
- [Docs] v1.67.4-stable by @ishaan-jaff in #10338
- Prisma Migrate - support setting custom migration dir by @krrishdholakia in #10336
- Fix: Prevent cache token overwrite by last chunk in streaming usage by @mdonaj in #10284
- [UI] Fixes for sessions on UI - ensure errors have a session and use 1 session for test key by @ishaan-jaff in #10342
- [UI QA Bug Fix] - Fix SSO Sign in flow by @ishaan-jaff in #10344
- [UI] Fix infinite Scroll on
Modelson Test Key Page by @ishaan-jaff in #10343 - [UI QA Fix] Fix width of the model_id on Models Page by @ishaan-jaff in #10345
- Fix - support azure dall e custom pricing by @krrishdholakia in #10339
- [Bug Fix] UI QA - Fix wildcard model test connection not working by @ishaan-jaff in #10347
- Litellm UI improvements 04 26 2025 p1 by @krrishdholakia in #10346
- [QA] Allow managing sessions with
litellm_session_idby @ishaan-jaff in #10348 - Handle more gemini tool calling edge cases + support bedrock 'stable-image-core' by @krrishdholakia in #10351
- [Feat] Add logging callback support for /moderations API by @ishaan-jaff in #10390
- [Reliability fix] Redis transaction buffer - ensure all redis queues are periodically flushed by @ishaan-jaff in #10393
- [Bug Fix] Responses API - fix for handling multiturn responses API sessions by @ishaan-jaff in #10415
- build(deps): bump axios, @docusaurus/core, @docusaurus/plugin-google-gtag, @docusaurus/plugin-ideal-image and @docusaurus/preset-classic in /docs/my-website by @dependabot in #10419
- docs: Fix link formatting in GitHub PR template by @user202729 in #10417
- docs: Improve documentation of phoenix logging by @user202729 in #10416
- [Feat Security] - Allow blocking web crawlers by @ishaan-jaff in #10420
- [Feat] Add support for using Bedrock Knowledge Bases with LiteLLM /chat/completions requests by @ishaan-jaff in #10413
- Revert "build(deps): bump axios, @docusaurus/core, @docusaurus/plugin-google-gtag, @docusaurus/plugin-ideal-image and @docusaurus/preset-classic in /docs/my-website" by @ishaan-jaff in #10421
- fix google studio url by @nonZero in #10095
- [New model] Add openai/computer-use-preview cost tracking / pricing by @ishaan-jaff in #10422
- fix(langsmith.py): respect langsmith batch size param by @krrishdholakia in #10411
- Support
x-litellm-api-keyheader param + allow key at max budget to call non-llm api endpoints by @krrishdholakia in #10392
New Contributors
- @mdonaj made their first contribution in #10284
- @user202729 made their first contribution in #10417
- @nonZero made their first contribution in #10095
Full Changelog: v1.67.4-nightly...v1.67.5-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.67.5-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 270.0 | 290.6566157251101 | 6.175800923917475 | 0.0 | 1848 | 0 | 232.84122400002616 | 2432.3238870000523 |
| Aggregated | Passed ✅ | 270.0 | 290.6566157251101 | 6.175800923917475 | 0.0 | 1848 | 0 | 232.84122400002616 | 2432.3238870000523 |
Contributors
nonZero, mdonaj, and 4 other contributors
Assets 4
v1.67.4-stable
What's Changed
- [Feat] Expose Responses API on LiteLLM UI Test Key Page by @ishaan-jaff in #10166
- [Bug Fix] Spend Tracking Bug Fix, don't modify in memory default litellm params by @ishaan-jaff in #10167
- Bug Fix - Responses API, Loosen restrictions on allowed environments for computer use tool by @ishaan-jaff in #10168
- [UI] Bug Fix, team model selector by @ishaan-jaff in #10171
- [Bug Fix] Auth Check, Fix typing to ensure case where model is None is handled by @ishaan-jaff in #10170
- [Docs] Responses API by @ishaan-jaff in #10172
- Litellm release notes 04 19 2025 by @krrishdholakia in #10169
- fix(transformation.py): pass back in gemini thinking content to api by @krrishdholakia in #10173
- Litellm docs SCIM by @ishaan-jaff in #10174
- fix(common_daily_activity.py): support empty entity id field by @krrishdholakia in #10175
- fix(proxy_server.py): pass llm router to get complete model list by @krrishdholakia in #10176
- Model pricing updates for Azure & VertexAI by @marty-sullivan in #10178
- fix(bedrock): wrong system prompt transformation by @hewliyang in #10120
- Fix: Potential SQLi in spend_management_endpoints.py by @n1lanjan in #9878
- Handle edge case where user sets model_group inside model_info + Return hashed_token in
tokenfield on/key/generateby @krrishdholakia in #10191 - Remove user_id from url by @krrishdholakia in #10192
- [Feat] Pass through endpoints - ensure
PassthroughStandardLoggingPayloadis logged and contains method, url, request/response body by @ishaan-jaff in #10194 - [Feat] Add Responses API - Routing Affinity logic for sessions by @ishaan-jaff in #10193
- [Feat] Add infinity embedding support (contributor pr) by @ishaan-jaff in #10196
- [Bug Fix] caching does not account for thinking or reasoning_effort config by @ishaan-jaff in #10140
- Gemini-2.5-flash improvements by @krrishdholakia in #10198
- Add AgentOps Integration to LiteLLM by @Dwij1704 in #9685
- Add global filtering to Users tab by @krrishdholakia in #10195
- [Feat] Add Support for DELETE /v1/responses/{response_id} on OpenAI, Azure OpenAI by @ishaan-jaff in #10205
- Bug Fix - Address deprecation of open_text by @ishaan-jaff in #10208
- UI - Users page - Enable global sorting (allows finding users with highest spend) by @krrishdholakia in #10211
- feat: Added Missing Attributes For Arize & Phoenix Integration (#10043) by @ishaan-jaff in #10215
- Users page - new user info pane by @krrishdholakia in #10213
- Fix datadog llm observability logging + (Responses API) Ensures handling for undocumented event types by @krrishdholakia in #10206
- Discard duplicate sentence by @DimitriPapadopoulos in #10231
- Require auth for all dashboard pages by @crisshaker in #10229
- [Feat] Add gpt-image-1 cost tracking by @ishaan-jaff in #10241
- [Bug Fix] Add Cost Tracking for gpt-image-1 when quality is unspecified by @ishaan-jaff in #10247
- [Feat] Add support for GET Responses Endpoint - OpenAI, Azure OpenAI by @ishaan-jaff in #10235
- fix(user_dashboard.tsx): add token expiry logic to user dashboard by @krrishdholakia in #10250
- [Helm] fix for serviceAccountName on migration job by @ishaan-jaff in #10258
- Fix typos by @DimitriPapadopoulos in #10232
- Reset key alias value when resetting filters by @crisshaker in #10099
- Support all compatible bedrock params when model="arn:.." by @krrishdholakia in #10256
- UI - fix edit azure public model name + support changing model names post create by @krrishdholakia in #10249
- Litellm fix UI login by @krrishdholakia in #10260
- Multi-admin + Users page fixes: show all models, show user personal models, allow editing user role, available models by @krrishdholakia in #10259
- Fix UI Flicker in Dashboard by @crisshaker in #10261
- Keys and tools pages: Use proper terminology for loading and no data cases by @msabramo in #10253
- adding support for cohere command-a-03-2025 by @ryanchase-cohere in #10295
- [Feat] Add GET, DELETE Responses endpoints on LiteLLM Proxy by @ishaan-jaff in #10297
- [Bug Fix] Timestamp Granularities are not properly passed to whisper in Azure by @ishaan-jaff in #10299
- Contributor PR - Support max_completion_tokens on Sagemaker (#10243) by @ishaan-jaff in #10300
- feat(grafana_dashboard): enable datasource selection via templating by @minatoaquaMK2 in #10257
- Update image_generation.md parameters by @daureg in #10312
- Update deprecation dates and prices by @o-khytrov in #10308
- Fix SSO user login - invalid token error by @krrishdholakia in #10298
- UI - Add team based filtering to models page by @krrishdholakia in #10325
- UI (Teams Page) - Support filtering by team id + team name by @krrishdholakia in #10324
- Move UI to encrypted token usage by @krrishdholakia in #10302
- add azure/gpt-image-1 pricing by @marty-sullivan in #10327
- fix(ui_sso.py): support experimental jwt keys for UI auth w/ SSO by @krrishdholakia in #10326
- UI (Keys Page) - Support cross filtering, filter by user id, filter by key hash by @krrishdholakia in #10322
- [Feat] Responses API - Add session management support for non-openai models by @ishaan-jaff in #10321
- Fix the table render on key creation. by @NANDINI-star in #10224
- Internal Users: Refresh user list on create by @crisshaker in #10296
- [Docs] UI Session Logs by @ishaan-jaff in #10334
New Contributors
- @Dwij1704 made their first contribution in #9685
- @DimitriPapadopoulos made their first contribution in #10231
- @ryanchase-cohere made their first contribution in #10295
- @minatoaquaMK2 made their first contribution in #10257
- @daureg made their first contribution in #10312
- @o-khytrov made their first contribution in #10308
Full Changelog: v1.67.0-stable...v1.67.4-stable
Contributors
daureg, msabramo, and 12 other contributors
Assets 2
v1.67.4-nightly
What's Changed
- Fix UI Flicker in Dashboard by @crisshaker in #10261
- Keys and tools pages: Use proper terminology for loading and no data cases by @msabramo in #10253
- adding support for cohere command-a-03-2025 by @ryanchase-cohere in #10295
- [Feat] Add GET, DELETE Responses endpoints on LiteLLM Proxy by @ishaan-jaff in #10297
- [Bug Fix] Timestamp Granularities are not properly passed to whisper in Azure by @ishaan-jaff in #10299
- Contributor PR - Support max_completion_tokens on Sagemaker (#10243) by @ishaan-jaff in #10300
- feat(grafana_dashboard): enable datasource selection via templating by @minatoaquaMK2 in #10257
- Update image_generation.md parameters by @daureg in #10312
- Update deprecation dates and prices by @o-khytrov in #10308
- Fix SSO user login - invalid token error by @krrishdholakia in #10298
- UI - Add team based filtering to models page by @krrishdholakia in #10325
- UI (Teams Page) - Support filtering by team id + team name by @krrishdholakia in #10324
- Move UI to encrypted token usage by @krrishdholakia in #10302
- add azure/gpt-image-1 pricing by @marty-sullivan in #10327
- fix(ui_sso.py): support experimental jwt keys for UI auth w/ SSO by @krrishdholakia in #10326
- UI (Keys Page) - Support cross filtering, filter by user id, filter by key hash by @krrishdholakia in #10322
- [Feat] Responses API - Add session management support for non-openai models by @ishaan-jaff in #10321
- Fix the table render on key creation. by @NANDINI-star in #10224
- Internal Users: Refresh user list on create by @crisshaker in #10296
- [Docs] UI Session Logs by @ishaan-jaff in #10334
New Contributors
- @ryanchase-cohere made their first contribution in #10295
- @minatoaquaMK2 made their first contribution in #10257
- @daureg made their first contribution in #10312
- @o-khytrov made their first contribution in #10308
Full Changelog: v1.67.3.dev1...v1.67.4-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.67.4-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.67.4-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 220.0 | 247.38248619018765 | 6.061326672784343 | 0.0 | 1814 | 0 | 197.54123199999185 | 2435.6727050000018 |
| Aggregated | Passed ✅ | 220.0 | 247.38248619018765 | 6.061326672784343 | 0.0 | 1814 | 0 | 197.54123199999185 | 2435.6727050000018 |
Contributors
daureg, msabramo, and 8 other contributors
Assets 4
v1.67.3.dev6
Full Changelog: v1.67.3.dev4...v1.67.3.dev6
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.67.3.dev6
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 240.0 | 263.90902888665886 | 6.165203372220349 | 0.0 | 1844 | 0 | 210.97686299992802 | 2930.0805719999516 |
| Aggregated | Passed ✅ | 240.0 | 263.90902888665886 | 6.165203372220349 | 0.0 | 1844 | 0 | 210.97686299992802 | 2930.0805719999516 |
v1.67.3.dev4
What's Changed
- Fix UI Flicker in Dashboard by @crisshaker in #10261
- Keys and tools pages: Use proper terminology for loading and no data cases by @msabramo in #10253
- adding support for cohere command-a-03-2025 by @ryanchase-cohere in #10295
- [Feat] Add GET, DELETE Responses endpoints on LiteLLM Proxy by @ishaan-jaff in #10297
- [Bug Fix] Timestamp Granularities are not properly passed to whisper in Azure by @ishaan-jaff in #10299
- Contributor PR - Support max_completion_tokens on Sagemaker (#10243) by @ishaan-jaff in #10300
- feat(grafana_dashboard): enable datasource selection via templating by @minatoaquaMK2 in #10257
- Update image_generation.md parameters by @daureg in #10312
- Update deprecation dates and prices by @o-khytrov in #10308
- Fix SSO user login - invalid token error by @krrishdholakia in #10298
- UI - Add team based filtering to models page by @krrishdholakia in #10325
- UI (Teams Page) - Support filtering by team id + team name by @krrishdholakia in #10324
- Move UI to encrypted token usage by @krrishdholakia in #10302
- add azure/gpt-image-1 pricing by @marty-sullivan in #10327
- fix(ui_sso.py): support experimental jwt keys for UI auth w/ SSO by @krrishdholakia in #10326
- UI (Keys Page) - Support cross filtering, filter by user id, filter by key hash by @krrishdholakia in #10322
- [Feat] Responses API - Add session management support for non-openai models by @ishaan-jaff in #10321
- Fix the table render on key creation. by @NANDINI-star in #10224
- Internal Users: Refresh user list on create by @crisshaker in #10296
- [Docs] UI Session Logs by @ishaan-jaff in #10334
- [Docs] v1.67.4-stable by @ishaan-jaff in #10338
- Prisma Migrate - support setting custom migration dir by @krrishdholakia in #10336
- Fix: Prevent cache token overwrite by last chunk in streaming usage by @mdonaj in #10284
New Contributors
- @ryanchase-cohere made their first contribution in #10295
- @minatoaquaMK2 made their first contribution in #10257
- @daureg made their first contribution in #10312
- @o-khytrov made their first contribution in #10308
- @mdonaj made their first contribution in #10284
Full Changelog: v1.67.3.dev1...v1.67.3.dev4
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.67.3.dev4
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 190.0 | 216.71376621493337 | 6.269037380681852 | 0.0 | 1875 | 0 | 164.64077799997767 | 4562.471842000036 |
| Aggregated | Passed ✅ | 190.0 | 216.71376621493337 | 6.269037380681852 | 0.0 | 1875 | 0 | 164.64077799997767 | 4562.471842000036 |
Contributors
daureg, msabramo, and 9 other contributors
Assets 4
v1.67.3.dev1
What's Changed
- [Feat] Add gpt-image-1 cost tracking by @ishaan-jaff in #10241
- [Bug Fix] Add Cost Tracking for gpt-image-1 when quality is unspecified by @ishaan-jaff in #10247
- [Feat] Add support for GET Responses Endpoint - OpenAI, Azure OpenAI by @ishaan-jaff in #10235
- fix(user_dashboard.tsx): add token expiry logic to user dashboard by @krrishdholakia in #10250
- [Helm] fix for serviceAccountName on migration job by @ishaan-jaff in #10258
- Fix typos by @DimitriPapadopoulos in #10232
- Reset key alias value when resetting filters by @crisshaker in #10099
- Support all compatible bedrock params when model="arn:.." by @krrishdholakia in #10256
- UI - fix edit azure public model name + support changing model names post create by @krrishdholakia in #10249
- Litellm fix UI login by @krrishdholakia in #10260
- Multi-admin + Users page fixes: show all models, show user personal models, allow editing user role, available models by @krrishdholakia in #10259
Full Changelog: v1.67.2-nightly...v1.67.3.dev1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.67.3.dev1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.67.3.dev1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 210.0 | 235.18092614107948 | 6.181088327781123 | 0.0 | 1850 | 0 | 192.45027600004505 | 4892.269687999942 |
| Aggregated | Passed ✅ | 210.0 | 235.18092614107948 | 6.181088327781123 | 0.0 | 1850 | 0 | 192.45027600004505 | 4892.269687999942 |
Contributors
DimitriPapadopoulos, krrishdholakia, and 2 other contributors
Assets 4
v1.67.2-nightly
What's Changed
- Add AgentOps Integration to LiteLLM by @Dwij1704 in #9685
- Add global filtering to Users tab by @krrishdholakia in #10195
- [Feat] Add Support for DELETE /v1/responses/{response_id} on OpenAI, Azure OpenAI by @ishaan-jaff in #10205
- Bug Fix - Address deprecation of open_text by @ishaan-jaff in #10208
- UI - Users page - Enable global sorting (allows finding users with highest spend) by @krrishdholakia in #10211
- feat: Added Missing Attributes For Arize & Phoenix Integration (#10043) by @ishaan-jaff in #10215
- Users page - new user info pane by @krrishdholakia in #10213
- Fix datadog llm observability logging + (Responses API) Ensures handling for undocumented event types by @krrishdholakia in #10206
- Discard duplicate sentence by @DimitriPapadopoulos in #10231
- Require auth for all dashboard pages by @crisshaker in #10229
New Contributors
Full Changelog: v1.67.1-nightly...v1.67.2-nightly
Contributors
DimitriPapadopoulos, krrishdholakia, and 3 other contributors
Assets 2
v1.67.1-nightly
What's Changed
- [UI] Bug Fix, team model selector by @ishaan-jaff in #10171
- [Bug Fix] Auth Check, Fix typing to ensure case where model is None is handled by @ishaan-jaff in #10170
- [Docs] Responses API by @ishaan-jaff in #10172
- Litellm release notes 04 19 2025 by @krrishdholakia in #10169
- fix(transformation.py): pass back in gemini thinking content to api by @krrishdholakia in #10173
- Litellm docs SCIM by @ishaan-jaff in #10174
- fix(common_daily_activity.py): support empty entity id field by @krrishdholakia in #10175
- fix(proxy_server.py): pass llm router to get complete model list by @krrishdholakia in #10176
- Model pricing updates for Azure & VertexAI by @marty-sullivan in #10178
- fix(bedrock): wrong system prompt transformation by @hewliyang in #10120
- Fix: Potential SQLi in spend_management_endpoints.py by @n1lanjan in #9878
- Handle edge case where user sets model_group inside model_info + Return hashed_token in
tokenfield on/key/generateby @krrishdholakia in #10191 - Remove user_id from url by @krrishdholakia in #10192
- [Feat] Pass through endpoints - ensure
PassthroughStandardLoggingPayloadis logged and contains method, url, request/response body by @ishaan-jaff in #10194 - [Feat] Add Responses API - Routing Affinity logic for sessions by @ishaan-jaff in #10193
- [Feat] Add infinity embedding support (contributor pr) by @ishaan-jaff in #10196
- [Bug Fix] caching does not account for thinking or reasoning_effort config by @ishaan-jaff in #10140
- Gemini-2.5-flash improvements by @krrishdholakia in #10198
Full Changelog: v1.67.0-nightly...v1.67.1-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.67.1-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 220.0 | 263.64700999835935 | 6.1132795166960605 | 0.0 | 1829 | 0 | 199.11094299999377 | 4358.182531000011 |
| Aggregated | Passed ✅ | 220.0 | 263.64700999835935 | 6.1132795166960605 | 0.0 | 1829 | 0 | 199.11094299999377 | 4358.182531000011 |
Contributors
n1lanjan, marty-sullivan, and 3 other contributors
Assets 4
v1.67.0-stable
What's Changed
- build(model_prices_and_context_window.json): add gpt-4.1 pricing by @krrishdholakia in #9990
- [Fixes/QA] For gpt-4.1 costs by @ishaan-jaff in #9991
- Fix cost for Phi-4-multimodal output tokens by @emerzon in #9880
- chore(docs): update ordering of logging & observability docs by @marcklingen in #9994
- Updated cohere v2 passthrough by @krrishdholakia in #9997
- [Feat] Add support for
cache_control_injection_pointsfor Anthropic API, Bedrock API by @ishaan-jaff in #9996 - [UI] Allow setting prompt
cache_control_injection_pointsby @ishaan-jaff in #10000 - Fix azure tenant id check from env var + response_format check on api_version 2025+ by @krrishdholakia in #9993
- Add
/vllmand/mistralpassthrough endpoints by @krrishdholakia in #10002 - CI/CD fix mock tests by @ishaan-jaff in #10003
- Setting
litellm.modify_paramsvia environment variables by @Eoous in #9964 - Support checking provider
/modelsendpoints on proxy/v1/modelsendpoint by @krrishdholakia in #9958 - Update AWS bedrock regions by @Schnitzel in #9430
- Fix case where only system messages are passed to Gemini by @NolanTrem in #9992
- Revert "Fix case where only system messages are passed to Gemini" by @krrishdholakia in #10027
- chore(docs): Update logging.md by @mrlorentx in #10006
- build(deps): bump @babel/runtime from 7.23.9 to 7.27.0 in /ui/litellm-dashboard by @dependabot in #10001
- Fix typo: Entrata -> Entra in code by @msabramo in #9922
- Retain schema field ordering for google gemini and vertex by @adrianlyjak in #9828
- Revert "Retain schema field ordering for google gemini and vertex" by @krrishdholakia in #10038
- Add aggregate team based usage logging by @krrishdholakia in #10039
- [UI Polish] UI fixes for cache control injection settings by @ishaan-jaff in #10031
- [UI] Bug Fix - Show created_at and updated_at for Users Page by @ishaan-jaff in #10033
- [Feat - Cost Tracking improvement] Track prompt caching metrics in DailyUserSpendTransactions by @ishaan-jaff in #10029
- Fix gcs pub sub logging with env var GCS_PROJECT_ID by @krrishdholakia in #10042
- Add property ordering for vertex ai schema (#9828) + Fix combining multiple tool calls by @krrishdholakia in #10040
- [Docs] Auto prompt caching by @ishaan-jaff in #10044
- Add litellm call id passing to Aim guardrails on pre and post-hooks calls by @hxmichael in #10021
- /utils/token_counter: get model_info from deployment directly by @chaofuyang in #10047
- [Bug Fix] Azure Blob Storage fixes by @ishaan-jaff in #10059
- build(deps): bump http-proxy-middleware from 2.0.7 to 2.0.9 in /docs/my-website by @dependabot in #10064
- fix(stream_chunk_builder_utils.py): don't set index on modelresponse by @krrishdholakia in #10063
- fix(llm_http_handler.py): fix fake streaming by @krrishdholakia in #10061
- Add aggregate spend by tag by @krrishdholakia in #10071
- Add OpenAI o3 & o4-mini by @PeterDaveHello in #10065
- Add new
/tag/daily/activityendpoint + Add tag dashboard to UI by @krrishdholakia in #10073 - Add team based usage dashboard at 1m+ spend logs (+ new
/team/daily/activityAPI) by @krrishdholakia in #10081 - [Feat SSO] Add LiteLLM SCIM Integration for Team and User management by @ishaan-jaff in #10072
- Virtual Keys: Filter by key alias (#10035) by @ishaan-jaff in #10085
- Add new
/vertex_ai/discoveryroute - enables calling AgentBuilder API routes by @krrishdholakia in #10084 - fix(o_series_transformation.py): correctly map o4 to openai o_series … by @krrishdholakia in #10079
- [Feat] Unified Responses API - Add Azure Responses API support by @ishaan-jaff in #10116
- UI: Make columns resizable/hideable in Models table by @msabramo in #10119
- Remove unnecessary
package*.jsonfiles by @msabramo in #10075 - Add Gemini Flash 2.5 Preview Model Price and Context Window by @drmingler in #10125
- test: update tests to new deployment model by @krrishdholakia in #10142
- [Feat] Support for all litellm providers on Responses API (works with Codex) - Anthropic, Bedrock API, VertexAI, Ollama by @ishaan-jaff in #10132
- fix(litellm-proxy-extras/utils.py): prisma migrate improvements: hand… by @krrishdholakia in #10138
- Litellm dev 04 18 2025 p2 by @krrishdholakia in #10157
- Gemini-2.5-flash - support reasoning cost calc + return reasoning content by @krrishdholakia in #10141
- Handle fireworks ai tool calling response by @krrishdholakia in #10130
- Support 'file' message type for VLLM video url's + Anthropic redacted message thinking support by @krrishdholakia in #10129
- fix(triton/completion/transformation.py): remove bad_words / stop wor… by @krrishdholakia in #10163
- Update model_prices_and_context_window_backup.json by @Classic298 in #10122
- to get API key from environment viarble of WATSONX_APIKEY by @ongkhaiwei in #10131
- test(utils.py): handle scenario where text tokens + reasoning tokens … by @krrishdholakia in #10165
New Contributors
- @Eoous made their first contribution in #9964
- @mrlorentx made their first contribution in #10006
- @hxmichael made their first contribution in #10021
- @chaofuyang made their first contribution in #10047
- @drmingler made their first contribution in #10125
- @Classic298 made their first contribution in #10122
- @ongkhaiwei made their first contribution in #10131
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.67.0-stable
Full Changelog: v1.66.0-stable...v1.67.0-stable
Contributors
msabramo, Schnitzel, and 15 other contributors
Assets 2
v1.67.0-nightly
What's Changed
- [Feat] Expose Responses API on LiteLLM UI Test Key Page by @ishaan-jaff in #10166
- [Bug Fix] Spend Tracking Bug Fix, don't modify in memory default litellm params by @ishaan-jaff in #10167
- Bug Fix - Responses API, Loosen restrictions on allowed environments for computer use tool by @ishaan-jaff in #10168
Full Changelog: v1.67.0-stable...v1.67.0-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.67.0-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 230.0 | 262.85419851041036 | 6.266552109647687 | 0.0 | 1873 | 0 | 202.24337799993464 | 5393.98836700002 |
| Aggregated | Passed ✅ | 230.0 | 262.85419851041036 | 6.266552109647687 | 0.0 | 1873 | 0 | 202.24337799993464 | 5393.98836700002 |
Contributors
ishaan-jaff