Include deferral in Task Instance Duration monitor

[Veldhoen]

Description

The Task Instance Duration monitor displays task duration, distinguishing queued duration and total duration.
However, we have a task that spends a lot of time in deferred state, and would like the monitor to display that too (in a similar fashion as the queing time).
Actually, after being deferred the task can get queued again, and I think the monitor currently shows the last time the task was queued, rather than the total time the task instance spent in queued state. In fact, I guess you could consider that a bug.

Use case/motivation

We have an external service that a task depends on. If the Airflow monitor can display deferred state duration, we would be able monitor the service's response time through the Airflow UI.

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[potiuk]

It might be diffifcult one to get and it's a feature - so it might only be implemented (if someone picks it up) in Airflow 3.1+

[tirkarthi]

There is queued_dttm and scheduled_dttm for a task instance in the database. Adding a similar deferred_dttm that is nullable and updated during deferral is one option.