Skip to content

feat(weave): Implement integration with Huggingface inference client #3612

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Feb 8, 2025

Conversation

soumik12345
Copy link
Contributor

@soumik12345 soumik12345 commented Feb 6, 2025

Description

Implement autopatch integration with 🤗 inference client (continuing from #2795).

Examples

LLMs

import os
import weave
from huggingface_hub import InferenceClient


weave.init("test-huggingface")
client = InferenceClient(api_key=os.getenv("HUGGINGFACE_API_KEY"))

image_url = "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
client.chat_completion(
    model="meta-llama/Llama-3.2-11B-Vision-Instruct",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "image_url", "image_url": {"url": image_url}},
                {"type": "text", "text": "Describe this image in one sentence."},
            ],
        }
    ],
    max_tokens=500,
)

Sample trace

Image generation

import os
import weave
from huggingface_hub import InferenceClient

weave.init("test-huggingface")
InferenceClient(api_key=os.getenv("HUGGINGFACE_API_KEY")).text_to_image(
    prompt="A cute puppy",
    model="black-forest-labs/FLUX.1-schnell",
    num_inference_steps=4,
)

Sample trace

@soumik12345 soumik12345 self-assigned this Feb 6, 2025
@soumik12345 soumik12345 requested a review from a team as a code owner February 6, 2025 04:56
@soumik12345 soumik12345 marked this pull request as draft February 6, 2025 04:57
@circle-job-mirror
Copy link

circle-job-mirror bot commented Feb 6, 2025

@soumik12345 soumik12345 marked this pull request as ready for review February 6, 2025 08:42
@soumik12345 soumik12345 requested a review from ayulockin February 6, 2025 08:43
@soumik12345
Copy link
Contributor Author

Hi @tssweeney, this PR is ready for review.

Copy link
Collaborator

@tssweeney tssweeney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔥 you da boss!

@soumik12345 soumik12345 enabled auto-merge (squash) February 8, 2025 15:35
@soumik12345 soumik12345 merged commit 5481add into master Feb 8, 2025
132 checks passed
@soumik12345 soumik12345 deleted the feat/hf-inference branch February 8, 2025 15:45
@github-actions github-actions bot locked and limited conversation to collaborators Feb 8, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants