[metricbeat][prometheus][query] add additional debug logging to simplify troubleshooting

[stefans-elastic]

Proposed commit message

See title

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works. Where relevant, I have used the stresstest.sh script to run them under stress conditions and race detector to verify their stability.
• I have added an entry in ./changelog/fragments using the changelog tool.

Disruptive User Impact

Author's Checklist

• [ ]

How to test this PR locally

Related issues

• Closes [Prometheus]: Improve Prometheus integration observability/debuggability integrations#15693

Use cases

Screenshots

Screenshot of logs from local test:

Log entries on the screenshot (in case more details are needed):

{"log.level":"debug","@timestamp":"2025-11-05T15:29:16.910+0200","log.logger":"prometheus.query.module","log.origin":{"function":"github.com/elastic/beats/v7/metricbeat/mb/module.(*metricSetWrapper).run","file.name":"module/wrapper.go","file.line":218},"message":"Starting metricSetWrapper[module=prometheus, name=query, host=localhost:9090]","service.name":"metricbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2025-11-05T15:29:16.910+0200","log.logger":"prometheus.query","log.origin":{"function":"github.com/elastic/beats/v7/metricbeat/module/prometheus/query.(*MetricSet).Fetch","file.name":"query/query.go","file.line":81},"message":"Running queries. There are 2 queries to run","service.name":"metricbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2025-11-05T15:29:16.910+0200","log.logger":"prometheus.query","log.origin":{"function":"github.com/elastic/beats/v7/metricbeat/module/prometheus/query.(*MetricSet).Fetch","file.name":"query/query.go","file.line":86},"message":"Running query. URL: http://localhost:9090/api/v1/query_not_found?query=sum%28rate%28prometheus_http_requests_total%5B1m%5D%29%29","service.name":"metricbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2025-11-05T15:29:16.911+0200","log.logger":"prometheus.query","log.origin":{"function":"github.com/elastic/elastic-agent-libs/transport/httpcommon.(*HTTPTransportSettings).RoundTripper.LoggingDialer.func1","file.name":"transport/logging.go","file.line":43},"message":"Completed dialing successfully","service.name":"metricbeat","network.transport":"tcp","server.address":"localhost:9090","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2025-11-05T15:29:16.918+0200","log.logger":"prometheus.query","log.origin":{"function":"github.com/elastic/beats/v7/metricbeat/module/prometheus/query.(*MetricSet).Fetch","file.name":"query/query.go","file.line":104},"message":"error received from prometheus endpoint http://localhost:9090/api/v1/query_not_found?query=sum%28rate%28prometheus_http_requests_total%5B1m%5D%29%29: Method Not Allowed\n","service.name":"metricbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2025-11-05T15:29:16.918+0200","log.logger":"prometheus.query","log.origin":{"function":"github.com/elastic/beats/v7/metricbeat/module/prometheus/query.(*MetricSet).Fetch","file.name":"query/query.go","file.line":86},"message":"Running query. URL: http://localhost:9090/api/v1/query?query=100","service.name":"metricbeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2025-11-05T15:29:16.933+0200","log.logger":"prometheus.query","log.origin":{"function":"github.com/elastic/beats/v7/metricbeat/module/prometheus/query.(*MetricSet).Fetch","file.name":"query/query.go","file.line":116},"message":"Fetched 1 metric(s) from http://localhost:9090/api/v1/query?query=100","service.name":"metricbeat","ecs.version":"1.6.0"}

Logs

[github-actions]

🤖 GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @stefans-elastic? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit
• backport-active-all is the label that automatically backports to all active branches.
• backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
• backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.