Description
Describe the bug
Environment
Kubernetes version: v1.33.0
Helm: used to deploy all components
Monitoring stack: kube-prometheus-stack (latest from prometheus-community/helm-charts)
Trivy Operator version: 0.26.1
kube-state-metrics version: latest as of May 2025
❗Issue 1: Kubernetes Storage Panels show “No data”
Affected Panels (across multiple dashboards):
Persistent Volumes - Capacity and usage in %
Persistent Volumes - Inodes
Persistent Volumes - Usage in bytes
Symptoms:
These panels remain empty (No data) despite having multiple bound PVCs and working metrics for other Kubernetes components.
What we tried:
Ensured kubelet.serviceMonitor.enabled: true is set.
Enabled custom endpoints in Prometheus values:
yaml
kubelet:
enabled: true
serviceMonitor:
enabled: true
https: true
scrapeTimeout: 30s
tlsConfig:
insecureSkipVerify: true
endpoints:
- port: https-metrics
path: /metrics
scheme: https
- port: https-metrics
path: /metrics/resource
scheme: https
Verified that ServiceMonitor exists and targets correct ports (10250).
Scraped /metrics/resource manually via curl but still not all metrics appear (e.g., no kubelet_volume_stats_*).
Confirmed PVCs are mounted and active.
❗Issue 2: Trivy CVE Panel does not show data
Dashboard Section:
Detailed CVE vulnerabilities in All namespace(s)
Symptoms:
The panel shows No data while trivy_image_vulnerabilities metrics exist and are correctly collected by Prometheus.
What we tried:
Enabled this critical flag in trivy-values.yaml:
yaml
metricsVulnIdEnabled: true
Verified Trivy metrics with:
curl -s http://localhost:8080/metrics | grep trivy_vulnerability_id
→ No output despite trivy_image_vulnerabilities being present.
Confirmed Trivy Helm values are correct and RBAC is applied.
CVE reports exist in kubectl get vulnerabilityreports -A.
How to reproduce?
No response
Expected behavior
No response
Additional context
No response