Pydantic settings cannot parse PREFECT_CLIENT_CUSTOM_HEADERS

[rekumar]

Bug summary

I'm trying to set a custom header for the prefect client using PREFECT_CLIENT_CUSTOM_HEADERS as per docs. This triggers a Pydantic validation error in the Settings class, as the json string is not being parsed to a dictionary for some reason.

Problem is current on my fork of branch 8c02ab1.

uv run prefect config set PREFECT_CLIENT_CUSTOM_HEADERS='{"CF-Access-Client-Id": "your-client-id", "CF-Access-Client-Secret": "your-secret"}' #exact line from the docs
Validation error(s) for setting PREFECT_CLIENT_CUSTOM_HEADERS

 - Input should be a valid dictionary

Also fails if I manually put this string into profiles.toml

Simple fix is to put a field_validator on ClientSettings.custom_headers to json.loads if receiving a string.

Version info

Version:             3.4.11.dev4+3.gb6f64e41da
API version:         0.8.4
Python version:      3.11.11
Git commit:          b6f64e41
Built:               Tue, Jul 29, 2025 10:02 PM
OS/Arch:             darwin/arm64
Profile:             local
Server type:         server
Pydantic version:    2.11.7

Additional context

No response

[rekumar]

The problem arises because Pydantic is only converting json strings -> dicts when reading from env variables.

with .env

PREFECT_CLIENT_CUSTOM_HEADERS = '{"hi": "bye"}'

I correctly get

rishi@rishibook prefect % uv run --env-file .env ipython
Python 3.11.11 (main, Jan 14 2025, 23:36:41) [Clang 19.1.6 ]
Type 'copyright', 'credits' or 'license' for more information
IPython 9.3.0 -- An enhanced Interactive Python. Type '?' for help.
Tip: Run your doctests from within IPython for development and debugging. The special %doctest_mode command toggles a mode where the prompt, output and exceptions display matches as closely as possible that of the default Python interpreter.

In [1]: from prefect.settings.models.client import ClientSettings

In [2]: ClientSettings()
Out[2]: ClientSettings(max_retries=5, retry_jitter_factor=0.2, retry_extra_codes=set(), csrf_support_enabled=True, custom_headers={'hi': 'bye'}, metrics=ClientMetricsSettings(enabled=False, port=4201))

But when you run prefect config set, the key/val provided in the CLI is directly passed into the Settings model here, and validation fails on strings. I assume the validation path for profiles.toml is similar.