Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set any groq model as default LLMClient for giskard #1977

Open
Vikas9758 opened this issue Jul 10, 2024 · 1 comment
Open

Set any groq model as default LLMClient for giskard #1977

Vikas9758 opened this issue Jul 10, 2024 · 1 comment
Labels
enhancement New feature or request good first issue Good for newcomers

Comments

@Vikas9758
Copy link

Vikas9758 commented Jul 10, 2024

🚀 Feature Request

Add Support for Groq in Giskard or show it in the documentation if already supported

🔈 Motivation

The integration of Groq as an alternative to OpenAI in the Giskard library is highly beneficial. Groq offers speed that is almost equivalent or even faster than GPT-4 and significantly faster than any Ollama model. This will provide users with a high-performance alternative for their machine learning tasks.

🛰 Alternatives

Currently, the only alternative is using OpenAI's models, which may not meet all users' needs for speed and performance. Can use Ollama as well but using it is a lot of hassle especially in colab or kaggle notebooks.Incorporating Groq provides a competitive edge and additional flexibility.

📎 Additional context

Here is the proposed code to integrate Groq with Giskard:

from groq import `Groq`
from giskard.llm.client.openai import OpenAIClient
import giskard as gsk

_client = Groq(api_key="YOUR_GROQ_API_KEY")
oc = OpenAIClient(model="any_model_in_groq", client=_client)
gsk.llm.set_default_client(oc)

This simple addition allows users to set up Groq as their default client in Giskard, enabling them to take advantage of Groq's superior speed and performance. Detailed documentation and examples will also be necessary to guide users through the setup and usage process.

@Vikas9758 Vikas9758 added the enhancement New feature or request label Jul 10, 2024
@Vikas9758 Vikas9758 changed the title Set any groq model as default llm client for giskard Set any groq model as default LLMClient for giskard Jul 11, 2024
@henchaves
Copy link
Member

Hey @Vikas9758, thanks for your suggestion!
Indeed it would be very useful to use Groq API itself instead of implementing a custom client.
Would you be interested in implementing this?

@henchaves henchaves added the good first issue Good for newcomers label Oct 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers
Development

No branches or pull requests

2 participants