feat: Allow OpenAI client config in `OpenAIChatGenerator` and `AzureOpenAIChatGenerator` #9215

Amnah199 · 2025-04-10T23:53:21Z

Related Issues

fixes Add passing http_client to OpenAI clients #9169

Proposed Changes:

Currently, users cannot configure the HTTP client used in these chat generators. To solve this:

add a http_client_kwargs param to store HTTP client configuration options. These options will be used in the init method to construct the client and can be safely serialized and deserialized.
Add a util function init_http_client which will be reused in configuring openai client in different components.
Unrelated but I took this chance to update the Azure embedders to use the new util function.

How did you test it?

Added new tests
Updated the existing tests

Notes for the reviewer

Similar to this PR

Checklist

I have read the contributors guidelines and the code of conduct
I have updated the related issue with new insights and changes
I added unit tests and updated the docstrings
I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
I documented my code
I ran pre-commit hooks and fixed any issue

anakin87 · 2025-04-11T08:30:10Z

Related community PR: feat: add passing http_client to components using OpenAI clients #9170
I've recently refactored the client creation method to also support async.
(At this point, it seems appropriate to refactor and move the client creation method in utils to reuse it.)

coveralls · 2025-04-11T11:53:05Z

Pull Request Test Coverage Report for Build 14497519261

Details

0 of 0 changed or added relevant lines in 0 files are covered.
15 unchanged lines in 4 files lost coverage.
Overall coverage decreased (-0.002%) to 90.372%

Files with Coverage Reduction	New Missed Lines	%
components/embedders/azure_document_embedder.py	2	96.0%
components/embedders/azure_text_embedder.py	2	95.56%
components/generators/chat/azure.py	4	92.73%
components/generators/chat/openai.py	7	96.07%

Totals
Change from base Build 14497303601:	-0.002%
Covered Lines:	10691
Relevant Lines:	11830

💛 - Coveralls

anakin87

I like the approach...

I suggest creating a test module in test/test_utils/test_http_client.py with a few tests: test_init_http_client, test_http_client_kwargs_type_validation, and test_http_client_kwargs_with_invalid_params.

After doing that, we might also remove the specific tests from the components - I'll leave that up to you.

alexengrig · 2025-04-12T15:42:49Z

haystack/components/generators/chat/azure.py

@@ -128,6 +131,8 @@ def __init__(  # pylint: disable=too-many-positional-arguments
            the schema provided in the `parameters` field of the tool definition, but this may increase latency.
        :param azure_ad_token_provider: A function that returns an Azure Active Directory token, will be invoked on
            every request.
+        :param http_client_kwargs:
+            A dictionary of keyword arguments to configure a custom httpx.Client.


Hi @Amnah199!
I want to add similar support to other components, and I think they all should have the same description. I suggest adding documentation similar to the following for :param http_client::

A dictionary of keyword arguments to configure a custom httpx.Client for your use case: proxies, authentication and other advanced functionalities of HTTPX.
You can set a proxy with basic authorization using the environment variables: HTTP_PROXY and HTTPS_PROXY, ALL_PROXY and NO_PROXY, for example HTTP_PROXY=http://user:[email protected]:8080.

A dictionary of keyword arguments to configure a custom `httpx.Client` for your use case: [proxies](https://www.python-httpx.org/advanced/proxies), [authentication](https://www.python-httpx.org/advanced/authentication) and other [advanced functionalities](https://www.python-httpx.org/advanced/clients) of HTTPX. You can set a proxy with basic authorization using [the environment variables](https://www.python-httpx.org/environment_variables): `HTTP_PROXY` and `HTTPS_PROXY`, `ALL_PROXY` and `NO_PROXY`, for example `HTTP_PROXY=http://user:[email protected]:8080`.

haystack/utils/http_client.py

alexengrig · 2025-04-12T15:51:57Z

haystack/utils/http_client.py

+        raise TypeError("The parameter 'http_client_kwargs' must be a dictionary.")
+    if async_client:
+        return httpx.AsyncClient(**http_client_kwargs)
+    return httpx.Client(**http_client_kwargs)


I suggest using openai.DefaultHttpxClient because it provides specific configurations from OpenAI (see here).

My 2 cents.

In case the user wants to provide their own client, I would leave them free to choose the configurations.

@anakin87 As per my understanding, DefaultAsyncHttpxClient does not limit the configuration options, just the default parameters are pre-configured to be optimal. The user you retains full configurability as with a regular httpx.AsyncClient.

While I understand this point, as a user I would find it unexpected to pass some client kwargs and then get a client that automatically sets others as well.

If we decide to go this path, I would clearly indicate this in all docstrings.

@alexengrig I see the merits in both approaches. However, we've decided to stick with httpx.Client for now to avoid introducing potential confusion from using a preconfigured client. This way, users have full transparency and control when creating and customizing their own client.

anakin87

I left 2 minor comments.

(There also are merge conflicts)

test/utils/test_http_client.py

anakin87 · 2025-04-15T12:59:53Z

haystack/utils/http_client.py

+        raise TypeError("The parameter 'http_client_kwargs' must be a dictionary.")
+    if async_client:
+        return httpx.AsyncClient(**http_client_kwargs)
+    return httpx.Client(**http_client_kwargs)


My 2 cents.

In case the user wants to provide their own client, I would leave them free to choose the configurations.

Amnah199 · 2025-04-16T11:45:14Z

test/utils/test_http_client.py

+    assert http_client.base_url == "https://example.com"
+
+
+def test_init_http_client_async():


I removed tests for init_http_client method calls from all components. I think these should be sufficient.

anakin87

Looks great.

Feel free to address my comment (or not 😄 )

haystack/components/embedders/azure_document_embedder.py

…om/deepset-ai/haystack into configure-openai-client-generators

Allow OpenAI client config in chat generator

9d3ac7c

github-actions bot added topic:tests type:documentation Improvements on the docs labels Apr 10, 2025

Amnah199 added 5 commits April 11, 2025 15:33

Add init_http_client as a util method

5bf23a0

Update azure chat gen

f9853de

Fix linting

b87d92c

Fix typing error

b416603

Fix tests

5b455f4

Remove unused httpx

dc5acd9

Amnah199 marked this pull request as ready for review April 11, 2025 11:59

Amnah199 requested review from a team as code owners April 11, 2025 11:59

Amnah199 requested review from dfokina and vblagoje and removed request for a team April 11, 2025 11:59

Amnah199 mentioned this pull request Apr 11, 2025

feat: add passing http_client to components using OpenAI clients #9170

Closed

7 tasks

anakin87 reviewed Apr 11, 2025

View reviewed changes

alexengrig reviewed Apr 12, 2025

View reviewed changes

Add http_client utils test

327e424

anakin87 reviewed Apr 15, 2025

View reviewed changes

Amnah199 added 4 commits April 16, 2025 01:25

Move tests to test_http_client

fecaab1

Reverting back to httpx.Client

841d623

Fix docstrings

02dcbce

Small fixes

de441ee

Amnah199 commented Apr 16, 2025

View reviewed changes

Amnah199 requested a review from anakin87 April 16, 2025 11:48

Merge branch 'main' into configure-openai-client-generators

17478e5

anakin87 approved these changes Apr 16, 2025

View reviewed changes

haystack/components/embedders/azure_document_embedder.py Outdated Show resolved Hide resolved

Amnah199 added 3 commits April 16, 2025 21:06

Fix docstrings

265a862

Merge branch 'configure-openai-client-generators' of https://github.c…

4cb3c5b

…om/deepset-ai/haystack into configure-openai-client-generators

Merge branch 'main' into configure-openai-client-generators

6ceca1c

Amnah199 merged commit 4986377 into main Apr 16, 2025
17 checks passed

Amnah199 deleted the configure-openai-client-generators branch April 16, 2025 16:32

sjrl assigned Amnah199 Apr 17, 2025

alexengrig mentioned this pull request Apr 18, 2025

feat: Allow OpenAI client config in other components #9270

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Allow OpenAI client config in `OpenAIChatGenerator` and `AzureOpenAIChatGenerator` #9215

feat: Allow OpenAI client config in `OpenAIChatGenerator` and `AzureOpenAIChatGenerator` #9215

Amnah199 commented Apr 10, 2025 •

edited

Loading

anakin87 commented Apr 11, 2025

coveralls commented Apr 11, 2025 •

edited

Loading

anakin87 left a comment

alexengrig Apr 12, 2025

alexengrig Apr 12, 2025

anakin87 Apr 15, 2025

Amnah199 Apr 15, 2025

anakin87 Apr 15, 2025

Amnah199 Apr 16, 2025

anakin87 left a comment

anakin87 Apr 15, 2025

Amnah199 Apr 16, 2025

anakin87 left a comment

		assert http_client.base_url == "https://example.com"


		def test_init_http_client_async():

feat: Allow OpenAI client config in OpenAIChatGenerator and AzureOpenAIChatGenerator #9215

feat: Allow OpenAI client config in OpenAIChatGenerator and AzureOpenAIChatGenerator #9215

Conversation

Amnah199 commented Apr 10, 2025 • edited Loading

Related Issues

Proposed Changes:

How did you test it?

Notes for the reviewer

Checklist

anakin87 commented Apr 11, 2025

coveralls commented Apr 11, 2025 • edited Loading

Pull Request Test Coverage Report for Build 14497519261

Details

💛 - Coveralls

anakin87 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anakin87 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anakin87 left a comment

Choose a reason for hiding this comment

feat: Allow OpenAI client config in `OpenAIChatGenerator` and `AzureOpenAIChatGenerator` #9215

feat: Allow OpenAI client config in `OpenAIChatGenerator` and `AzureOpenAIChatGenerator` #9215

Amnah199 commented Apr 10, 2025 •

edited

Loading

coveralls commented Apr 11, 2025 •

edited

Loading