From 99b0ead14069963e546860261a5dba99f969c332 Mon Sep 17 00:00:00 2001 From: Andreas Tellenbach Date: Mon, 29 Jul 2024 09:27:56 +0200 Subject: [PATCH] Improve watchdog alert This expression fires the Watchdog alert also if the TSDB is up to date and therefore checks the functionality of the full stack. With the former `vector(1)`, only alertmanager needs to be functional to fire. So In case of a full TSDB storage, the Watchdog still fires and the lack of new metrics goes unnoticed. --- .../kube-prometheus/components/mixin/alerts/general.libsonnet | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/jsonnet/kube-prometheus/components/mixin/alerts/general.libsonnet b/jsonnet/kube-prometheus/components/mixin/alerts/general.libsonnet index c236283747..1cb484dcec 100644 --- a/jsonnet/kube-prometheus/components/mixin/alerts/general.libsonnet +++ b/jsonnet/kube-prometheus/components/mixin/alerts/general.libsonnet @@ -28,7 +28,7 @@ "DeadMansSnitch" integration in PagerDuty. |||, }, - expr: 'vector(1)', + expr: 'present_over_time(prometheus_tsdb_head_max_time[1m]) != 0', labels: { severity: 'none', },