Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

⚡️ Speed up method HuggingFaceInferenceAPIEmbeddingsComponent.build_embeddings by 40% in PR #6140 (API_Modal) #6626

Open
wants to merge 1 commit into
base: API_Modal
Choose a base branch
from
+18 −8
Open

⚡️ Speed up method HuggingFaceInferenceAPIEmbeddingsComponent.build_embeddings by 40% in PR #6140 (API_Modal) #6626

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 18 additions & 8 deletions src/backend/base/langflow/components/embeddings/huggingface_inference_api.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
from functools import cache
from urllib.parse import urlparse

import requests
Expand Down Expand Up @@ -46,6 +47,7 @@
Output(display_name="Embeddings", name="embeddings", method="build_embeddings"),
]

@cache

Check failure on line 50 in src/backend/base/langflow/components/embeddings/huggingface_inference_api.py

View workflow job for this annotation

GitHub Actions / Ruff Style Check (3.12)

Ruff (B019) 

src/backend/base/langflow/components/embeddings/huggingface_inference_api.py:50:5: B019 Use of `functools.lru_cache` or `functools.cache` on methods can lead to memory leaks
def validate_inference_endpoint(self, inference_endpoint: str) -> bool:
parsed_url = urlparse(inference_endpoint)
if not all([parsed_url.scheme, parsed_url.netloc]):
Expand All @@ -68,7 +70,7 @@
if response.status_code != requests.codes.ok:
msg = f"HuggingFace health check failed: {response.status_code}"
raise ValueError(msg)
# returning True to solve linting error

return True

def get_api_url(self) -> str:
Expand All @@ -90,17 +92,25 @@
or "huggingface.co" not in api_url.lower()
)

if not self.api_key and is_local_url:
self.validate_inference_endpoint(api_url)
api_key = SecretStr("APIKeyForLocalDeployment")
elif not self.api_key:
msg = "API Key is required for non-local inference endpoints"
raise ValueError(msg)
if api_url not in self.validated_endpoints:
if not self.api_key and is_local_url:
self.validate_inference_endpoint(api_url)
self.validated_endpoints.add(api_url)
api_key = SecretStr("APIKeyForLocalDeployment")
elif not self.api_key:
msg = "API Key is required for non-local inference endpoints"
raise ValueError(msg)
else:
api_key = SecretStr(self.api_key)
else:
api_key = SecretStr(self.api_key).get_secret_value()
api_key = SecretStr("APIKeyForLocalDeployment") if is_local_url else SecretStr(self.api_key)

try:
return self.create_huggingface_embeddings(api_key, api_url, self.model_name)
except Exception as e:
msg = "Could not connect to HuggingFace Inference API."
raise ValueError(msg) from e

def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.validated_endpoints = set()
Loading