File tree 1 file changed +11
-3
lines changed
1 file changed +11
-3
lines changed Original file line number Diff line number Diff line change @@ -91,9 +91,17 @@ $ litellm --model huggingface/bigcode/starcoder
91
91
92
92
### Step 2: Replace openai base
93
93
``` python
94
- import openai
95
- client = openai.OpenAI(api_key = " anything" , base_url = " http://0.0.0.0:8000" )
96
- print (openai.chat.completions.create(model = " test" , messages = [{" role" :" user" , " content" :" Hey!" }]))
94
+ import openai # openai v1.0.0+
95
+ client = openai.OpenAI(api_key = " anything" ,base_url = " http://0.0.0.0:8000" ) # set proxy to base_url
96
+ # request sent to model set on litellm proxy, `litellm --model`
97
+ response = client.chat.completions.create(model = " gpt-3.5-turbo" , messages = [
98
+ {
99
+ " role" : " user" ,
100
+ " content" : " this is a test request, write a short poem"
101
+ }
102
+ ])
103
+
104
+ print (response)
97
105
```
98
106
99
107
## Logging Observability ([ Docs] ( https://docs.litellm.ai/docs/observability/callbacks ) )
You can’t perform that action at this time.
0 commit comments