Skip to content

KubernetesExecutor shadowing timeout_seconds and _request_seconds #45517

Closed
@insomnes

Description

@insomnes

Apache Airflow Provider(s)

cncf-kubernetes

Versions of Apache Airflow Providers

apache-airflow-providers-cncf-kubernetes==10.0.1

Apache Airflow version

2.10.4

Operating System

Debian Bookworm

Deployment

Other

Deployment details

It's deployment independent and related to code directly

What happened

kube client query options of timeout_seconds and _request_seconds provided via kube_client_request_args config option are re-written via hardcoded values in executor code of KubernetesJobWatcher.

What you think should happen instead

Values set via AIRFLOW__KUBERNETES_EXECUTOR__KUBE_CLIENT_REQUEST_ARGS for timeout_seconds and _request_seconds should be respected, hardcoded values should be provided only in case of missing keys in the provided JSON.

Simple if for checking if key is already present would be enough

How to reproduce

  1. Set the kube_client_request_args option with JSON: {"timeout_seconds": 1800, "_request_timeout": 60}
  2. Run airflow scheduler with kubernetes executor configured
  3. In logs you can see that without pod events the KubernetesJobWatcher will still die every 30 seconds instead of 60.

Anything else

The problem was introduced with:
610747d#diff-d884d637ab746b1301ce80b30ba2c1908299ba6f13edcad6535c837fb4d30938R151

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions