Support List[str] in InferenceClient.feature_extraction text parameter

[anakin87]

Hello and thanks for your work!

Context

I work on the Haystack LLM framework, which provides Embedders based on huggingface_hub.
The HuggingFaceAPITextEmbedder transforms a string into a vector. The HuggingFaceAPIDocumentEmbedder enriches a Document with its embedding.

These components allow using different HF APIs via huggingface_hub: Serverless Inference API, Inference Endpoints, and self-hosted Text Embeddings Inference container.

In the past, we used InferenceClient.post to query the API, mainly to support truncate and normalize (which were missing in the feature_extraction method before #2270).

Now that InferenceClient.post is changing and will be removed in 0.31.0, I'm migrating to InferenceClient.feature_extraction.

Issue

There's only one issue:

• In HuggingFaceAPIDocumentEmbedder, we allow batching, which is explicitly supported by TEI (and Inference Endpoints).

• Passing a list of inputs to the text parameter of feature_extraction seems to work correctly—I've tested it with Serverless Inference API, Inference Endpoints, and self-hosted TEI.

• However, the method signature of feature_extraction officially supports only str, not List[str].

Questions

Given this, I'd like to ask:

• Do you plan to support List[str] in feature_extraction?
• Will this behavior continue to work in the future?
• If not, what alternative approach would you recommend for batching embedding generation via InferenceClient?

[Wauplin]

Hi @anakin87 , thanks for opening this issue and for describing your findings. I agree with you that supporting List[str] would be a great addition here. I've just opened huggingface/huggingface.js#1166 so that our official spec gets updated. Once merged, I'll make the change in huggingface_hub too.

For the record and as mentioned in the PR above, I have pushed back in the past for this feature request (see #1745 and #1746 (comment)) but I think it's time to revisit. I'll let you know once it's done but you can't already consider it safe and future-proof to use it like this.

EDIT: and glad to see the deprecation warning is been carefully listened too ❤️