Add Apache Airflow Monitoring #12072
Conversation
|
With #12076 gets merged, you should have the expected menu. Then, you should fix all CI tasks. UI submodule should be reverted. |
|
SWIP and relative updates are missing. |
oap-server/server-starter/src/main/resources/ui-initialized-templates/menu.yaml
Outdated
Show resolved
Hide resolved
You still don't fix the conflicts, CI would not run until you fix it. |
There was a problem hiding this comment.
The doc and code styles usually have history reason, only few are typos.
You should not randomly change them so randomly.
And if there's typo, it should not be fixed in a feature PR, it will be very confused when review git changelogs in the future.
Don't hurry in changing.
docs/en/changes/changes.md
Outdated
| @@ -130,5 +135,5 @@ | |||
| * Remove `OpenTelemetry Exporter` support from meter doc, as this has been flagged as unmaintained on OTEL upstream. | |||
| * Add doc of one-line quick start script for different storage types. | |||
| * Add FAQ for `Why is Clickhouse or Loki or xxx not supported as a storage option?`. | |||
|
|
|||
| * Add Airflow monitoring. | |||
There was a problem hiding this comment.
| * Add Airflow monitoring. | |
| * Add Airflow monitoring docs. | |
oap-server/server-starter/src/main/resources/hierarchy-definition.yml
Outdated
Show resolved
Hide resolved
|
And, you still don't resolve conflicts. Learn git more please. |
...ver/server-starter/src/main/resources/ui-initialized-templates/airflow/airflow-instance.json
Outdated
Show resolved
Hide resolved
songzhendong
force-pushed
the
master
branch
3 times, most recently
from
23206a5 to
40123ee
Compare
|
Don't close a PR, as you only update things. |
docs/en/changes/changes.md
Outdated
| * Update tabs of the Kubernetes service page. | ||
| * Support Airflow monitoring. | ||
| * Update tabs of the Kubernetes service page. |
There was a problem hiding this comment.
| * Update tabs of the Kubernetes service page. | |
| * Support Airflow monitoring. | |
| * Update tabs of the Kubernetes service page. | |
| * Update tabs of the Kubernetes service page. | |
| * Support Airflow monitoring. |
Duplicated
|
Please recheck UI submodule, you are changing that unexpectedly. |
| "aggregate_labels(meter_airflow_dag_processing_file_path_queue_size,sum))", | ||
| "aggregate_labels(meter_airflow_dag_processing_processes,sum))" |
There was a problem hiding this comment.
| "aggregate_labels(meter_airflow_dag_processing_file_path_queue_size,sum))", | |
| "aggregate_labels(meter_airflow_dag_processing_processes,sum))" | |
| "aggregate_labels(meter_airflow_dag_processing_file_path_queue_size,sum)", | |
| "aggregate_labels(meter_airflow_dag_processing_processes,sum)" | |
| - SW_STORAGE=h2 | ||
| # OpenTelemetry collector 1 | ||
| otel-collector: | ||
| image: otel/opentelemetry-collector-contrib:0.96.0 |
There was a problem hiding this comment.
| image: otel/opentelemetry-collector-contrib:0.96.0 | |
| image: otel/opentelemetry-collector:${OTEL_COLLECTOR_VERSION} |
There was a problem hiding this comment.
We'd better use a unified version.
| operations: | ||
| - action: add_label | ||
| new_label: service_instance_id # Add 'service_instance_id' label to metrics | ||
| new_value: "instance1" |
There was a problem hiding this comment.
When airflow metrics use the push mode, do the metrics have any labels to identify which service and instance?
You add labels manually here, I'm not sure when 1 otel-collector facing 2 airflow instances, how to transform the metrics and add labels.
There was a problem hiding this comment.
If this is impossible(if there is no value to have metadata in OTLP push), I think we should recommend using K8s sidecar as recommended deployment to add this label. In there, we could use pod name as the instance name, and k8s service name as the service name.
Meanwhile, is host_name representing the service name?
| # This file is used to show how to write configuration files and can be used to test. | ||
|
|
||
| setup: | ||
| env: compose |
There was a problem hiding this comment.
Please make sure whether compose is a good way to run metadata clear deployment. If it isn't, airflow instance + otel collector as sidecar deployment on k8s may be a better way.
e2e is not just for testing, you could see on the docs(you wrote), all configurations here are a kind of guidance and reference for users to deploy this.
| operations: | ||
| - action: add_label | ||
| new_label: host_name # Add 'host_name' label indicating the host name | ||
| new_value: "airflow-webserver.airflow" |
There was a problem hiding this comment.
If this is the service name, and hard coded instance name, I doubt the service and instance hierarchy would work.
I saw your screenshot, but that seems not match your e2e.
|
FYI @songzhendong We will update the MAL implementation for #11992. Then UI side will update to support multiple-label tags as well. You will need to update this PR accordingly, most about UI dashboard setup and e2e. |
Get |
|
No update for a week. @songzhendong If you want to continue to make this ready to merge, please let me know. |
Add Apache Airflow Monitoring
- Documentation has been updated in the
docsdirectory.- Unit tests and integration tests have been added under
testdirectory.CHANGESlog.