Inference endpoints no longer update non-secret environment variables.

[shopigarner]

Describe the bug

It appears somewhat recently that creating new and updating existing inference endpoints no longer create/update non-secret environment variables on the endpoint.

The documentation suggests setting "env" in the custom_image parameter. However when fetching the raw data from the HF endpoint, the "env" vars are directly under the model parameter, not under image.custom which is where the documentation suggests it should be.

There is a workaround to set the environment variable as a secret, which works just fine. Perhaps another workaround would be to just use requests and set the payload accordingly to match what's expected.

I'm not so sure this is an issue with this library as it is an issue in the API itself. Something appears to have changed on the inference endpoint API.

Examples

This does not appear to work:

hf_endpoint = update_inference_endpoint(
          name=endpoint_name,
          namespace="my_namespace",
          repository=model,
          revision=target_revision,
          custom_image={"env":{"MY_PARAMETER": "foo"}}
      )

The workaround:

hf_endpoint = update_inference_endpoint(
          name=endpoint_name,
          namespace="my_namespace",
          repository=model,
          revision=target_revision,
          secrets={"MY_PARAMETER": "foo"}
      )

When pulling the raw endpoint, we can see env is now under model and not image.custom:

{
  "model": {
    "repository": "MY/REPO",
    "revision": "asdfqwerty",
    "task": "text-generation",
    "framework": "pytorch",
    "image": {
      "custom": {
        "healthRoute": "/health",
        "port": 7080,
        "url": "my_image",
      }
    },
    "secrets": {
      "MY_SECRET": null
    },
    "env": {
      "MY_ENV": "true"
    }
  }
}

Reproduction

hf_endpoint = update_inference_endpoint(
          name=endpoint_name,
          namespace="my_namespace",
          repository=model,
          revision=target_revision,
          custom_image={"env":{"MY_PARAMETER": "foo"}}
      )

Logs

System info

- huggingface_hub version: 0.27.1
- Platform: macOS-15.3.2-arm64-arm-64bit
- Python version: 3.10.13

[Wauplin]

Thanks @shopigarner for the report.

cc @ErikKaum could you have a look at this please?