Skip to content
Discussion options

You must be logged in to vote

Yep! This is best done by passing in a client registry that replaces your primary LLM with an openai client where base_url points at a local OpenAI-compatible fake server.

If you have followup questions, please keep in mind that we do not use GH discussions for questions: we do not monitor this channel actively and are most responsive on our Discord server.

Replies: 1 comment

Comment options

You must be logged in to vote
0 replies
Answer selected by sxlijin
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
2 participants