v1.52.0.dev11

What's Changed

• (DB fix) don't run apply_db_fixes on startup by @ishaan-jaff in #6604

Full Changelog: v1.52.0...v1.52.0.dev11

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.0.dev11

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	250.0	291.82137058342585	6.047321226858995	0.0	1810	0	225.1587829999835	2676.3882499999454
Aggregated	Passed ✅	250.0	291.82137058342585	6.047321226858995	0.0	1810	0	225.1587829999835	2676.3882499999454

v1.52.0.dev8

Full Changelog: v1.52.0.dev6...v1.52.0.dev8

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.0.dev8

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	250.0	275.9906719829581	6.082205864218583	0.0	1819	0	217.16473999998698	1730.627453000011
Aggregated	Passed ✅	250.0	275.9906719829581	6.082205864218583	0.0	1819	0	217.16473999998698	1730.627453000011

v1.52.0.dev6

What's Changed

• (DB fix) don't run apply_db_fixes on startup by @ishaan-jaff in #6604

Full Changelog: v1.52.0...v1.52.0.dev6

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.0.dev6

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	200.0	225.40039438791652	6.306935058574255	0.0	1887	0	183.29160000007505	2583.1958680000184
Aggregated	Passed ✅	200.0	225.40039438791652	6.306935058574255	0.0	1887	0	183.29160000007505	2583.1958680000184

v1.52.0.dev4

Full Changelog: v1.51.3.dev10...v1.52.0.dev4

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.0.dev4

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	250.0	297.24728035064066	6.243440755366903	0.0	1868	0	230.16010100002404	2892.1668609999642
Aggregated	Passed ✅	250.0	297.24728035064066	6.243440755366903	0.0	1868	0	230.16010100002404	2892.1668609999642

v1.52.0.dev1

Full Changelog: v1.51.3.dev10...v1.52.0.dev1

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.0.dev1

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	220.0	247.31993145342028	6.2049107222098225	0.0	1857	0	200.41075999995428	3332.4286190000407
Aggregated	Passed ✅	220.0	247.31993145342028	6.2049107222098225	0.0	1857	0	200.41075999995428	3332.4286190000407

v1.52.0

What's Changed

• LiteLLM Minor Fixes & Improvements (11/01/2024) by @krrishdholakia in #6551
• Litellm dev 11 02 2024 by @krrishdholakia in #6561
• build(deps): bump cookie and express in /docs/my-website by @dependabot in #6566
• docs(virtual_keys.md): update Dockerfile reference by @emmanuel-ferdman in #6554
• (proxy fix) - call connect on prisma client when running setup by @ishaan-jaff in #6534
• Add 3.5 haiku by @paul-gauthier in #6588
• Litellm perf improvements 3 by @krrishdholakia in #6573
• (fix) /image/generation - ImageObject conversion when content_filter_results exists by @ishaan-jaff in #6584
• (fix) litellm.text_completion raises a non-blocking error on simple usage by @ishaan-jaff in #6546
• (feat) add Predicted Outputs for OpenAI by @ishaan-jaff in #6594
• (fix) Vertex Improve Performance when using image_url by @ishaan-jaff in #6593
• fix(lowest_tpm_rpm_routing.py): fix parallel rate limit check by @krrishdholakia in #6577

New Contributors

• @emmanuel-ferdman made their first contribution in #6554

Full Changelog: v1.51.3...v1.52.0

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.0

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	210.0	231.0704959909717	6.291122309918775	0.0	1883	0	180.74613400000317	2618.3897060000163
Aggregated	Passed ✅	210.0	231.0704959909717	6.291122309918775	0.0	1883	0	180.74613400000317	2618.3897060000163

v1.51.3.dev10

What's Changed

• LiteLLM Minor Fixes & Improvements (11/01/2024) by @krrishdholakia in #6551
• Litellm dev 11 02 2024 by @krrishdholakia in #6561
• build(deps): bump cookie and express in /docs/my-website by @dependabot in #6566
• docs(virtual_keys.md): update Dockerfile reference by @emmanuel-ferdman in #6554
• (proxy fix) - call connect on prisma client when running setup by @ishaan-jaff in #6534
• Add 3.5 haiku by @paul-gauthier in #6588
• Litellm perf improvements 3 by @krrishdholakia in #6573
• (fix) /image/generation - ImageObject conversion when content_filter_results exists by @ishaan-jaff in #6584
• (fix) litellm.text_completion raises a non-blocking error on simple usage by @ishaan-jaff in #6546
• (feat) add Predicted Outputs for OpenAI by @ishaan-jaff in #6594
• (fix) Vertex Improve Performance when using image_url by @ishaan-jaff in #6593

New Contributors

• @emmanuel-ferdman made their first contribution in #6554

Full Changelog: v1.51.3...v1.51.3.dev10

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.51.3.dev10

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	200.0	226.3206167901752	6.259170593216401	0.0	1873	0	186.16014400004133	1927.3238910000146
Aggregated	Passed ✅	200.0	226.3206167901752	6.259170593216401	0.0	1873	0	186.16014400004133	1927.3238910000146

v1.51.3-dev2

What's Changed

• LiteLLM Minor Fixes & Improvements (11/01/2024) by @krrishdholakia in #6551
• Litellm dev 11 02 2024 by @krrishdholakia in #6561

Full Changelog: v1.51.3...v1.51.3-dev2

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.51.3-dev2

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	210.0	234.0498504636111	6.1976137143687975	0.0	1855	0	187.64690299997255	3654.997406000007
Aggregated	Passed ✅	210.0	234.0498504636111	6.1976137143687975	0.0	1855	0	187.64690299997255	3654.997406000007

v1.51.3-dev1

What's Changed

• LiteLLM Minor Fixes & Improvements (11/01/2024) by @krrishdholakia in #6551
• Litellm dev 11 02 2024 by @krrishdholakia in #6561

Full Changelog: v1.51.3...v1.51.3-dev1

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.51.3-dev1

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	250.0	292.3714877928421	6.163980651581093	0.0	1844	0	226.11442700002726	2207.691740000001
Aggregated	Passed ✅	250.0	292.3714877928421	6.163980651581093	0.0	1844	0	226.11442700002726	2207.691740000001

v1.51.3

What's Changed

• Support specifying exponential backoff retry strategy when calling completions() by @dbczumar in #6520
• (fix) slack alerting - don't spam the failed cost tracking alert for the same model by @ishaan-jaff in #6543
• (feat) add XAI ChatCompletion Support by @ishaan-jaff in #6373
• LiteLLM Minor Fixes & Improvements (10/30/2024) by @krrishdholakia in #6519

Full Changelog: v1.51.2...v1.51.3

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.51.3

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	200.0	220.3819331893052	6.253936592654308	0.0	1870	0	179.7343989999831	3185.1700670000014
Aggregated	Passed ✅	200.0	220.3819331893052	6.253936592654308	0.0	1870	0	179.7343989999831	3185.1700670000014