Add support for Strimzi Metrics Reporter

This patch adds support for metrics types configuration, integrating the Strimzi Metrics Reporter, and custom Prometheus JMX Exporter configuration.
The custom configuration feature is needed for Cluster Operator integration, but it can also be leveraged by standalone users.
See Proposal 43 for more details.

Signed-off-by: Federico Valeri <[email protected]>

CHANGELOG.md

@@ -9,6 +9,15 @@
   * Both the `/openapi` and `/openapi/v3` endpoints return the OpenAPI v3 definition of the bridge REST API.
 * Removed script to build bridge configuration within the container. 
   It is going to be set up by the Strimzi operator within a ConfigMap and mounted as volume on the bridge pod.
+* Added support for [Strimzi Metrics Reporter](https://github.com/strimzi/metrics-reporter).
+  This is a Kafka plugin that directly exposes metrics in Prometheus format without using JMX, and can be enabled by setting `bridge.metrics=strimziMetricsReporter`.
+* Added support for custom Prometheus JMX Exporter configuration.
+  Set `bridge.metrics.exporter.config.path=/path/to/my-exporter-config.yml` to use your custom config.
+
+### Changes, deprecations and removals
+
+* `KAFKA_BRIDGE_METRICS_ENABLED` configuration has been deprecated.
+  Set `bridge.metrics=jmxPrometheusExporter` to keep using Prometheus JMX Exporter.
 
 ## 0.31.1
 

config/application.properties

@@ -1,5 +1,16 @@
 #Bridge related settings
 bridge.id=my-bridge
+
+# uncomment the following line to enable JMX Prometheus Exporter, check the documentation for more details
+#bridge.metrics=jmxPrometheusExporter
+# optionally, you can set the file path of your custom configuration
+#bridge.metrics.exporter.config.path=/path/to/my-exporter-config.yaml
+
+# uncomment the following line to enable Strimzi Metrics Reporter, check the documentation for more details
+#bridge.metrics=strimziMetricsReporter
+# optionally, you can filter the exposed metrics using a comma separated list of regexes
+#kafka.prometheus.metrics.reporter.allowlist=.*
+
 # uncomment the following line (bridge.tracing) to enable OpenTelemetry tracing, check the documentation for more details
 #bridge.tracing=opentelemetry
 

documentation/assemblies/assembly-kafka-bridge-config.adoc

@@ -8,11 +8,13 @@
 [role="_abstract"]
 Configure a deployment of the Kafka Bridge using configuration properties.
 Configure Kafka and specify the HTTP connection details needed to be able to interact with Kafka.
+It is possible to enable metrics in Prometheus format with JMX Prometheus Exporter or Strimzi Metrics Reporter.
 You can also use configuration properties to enable and use distributed tracing with the Kafka Bridge.
 Distributed tracing allows you to track the progress of transactions between applications in a distributed system.
 
 NOTE: Use the `KafkaBridge` resource to configure properties when you are xref:overview-components-running-kafka-bridge-cluster-{context}[running the Kafka Bridge on Kubernetes].
 
 include::modules/proc-configuring-kafka-bridge.adoc[leveloffset=+1]
-include::modules/proc-configuring-kafka-bridge-metrics.adoc[leveloffset=+1]
-include::modules/proc-configuring-kafka-bridge-tracing.adoc[leveloffset=+1]
+include::modules/proc-configuring-kafka-bridge-jmx-metrics.adoc[leveloffset=+1]
+include::modules/proc-configuring-kafka-bridge-smr-metrics.adoc[leveloffset=+1]
+include::modules/proc-configuring-kafka-bridge-tracing.adoc[leveloffset=+1]

documentation/modules/proc-configuring-kafka-bridge-jmx-metrics.adoc

@@ -0,0 +1,33 @@
+[id='proc-configuring-kafka-bridge-jmx-metrics-{context}']
+= Configuring JMX Exporter metrics
+
+[role="_abstract"]
+Enable metrics for the Kafka Bridge by setting the `bridge.metrics` configuration.
+
+.Prerequisites
+
+* xref:proc-downloading-kafka-bridge-{context}[The Kafka Bridge installation archive is downloaded].
+
+.Procedure
+
+. Set the `bridge.metrics` configuration to `jmxPrometheusExporter`.
++
+.Configuration for enabling metrics
+
+[source,properties]
+----
+bridge.metrics=jmxPrometheusExporter
+----
++
+Optionally, you can set a custom JMX Exporter configuration using the `bridge.metrics.exporter.config.path` property.
+If not set, a default embedded configuration file will be used.
+
+. Run the Kafka Bridge script to enable metrics.
++
+.Running the Kafka Bridge to enable metrics
+[source,shell]
+----
+./bin/kafka_bridge_run.sh --config-file=<path>/application.properties
+----
++
+With metrics enabled, you can use `GET /metrics` with the `/metrics` endpoint to retrieve Kafka Bridge metrics in Prometheus format.

documentation/modules/proc-configuring-kafka-bridge-smr-metrics.adoc

@@ -0,0 +1,36 @@
+[id='proc-configuring-kafka-bridge-smr-metrics-{context}']
+= Configuring Strimzi Metrics Reporter metrics
+
+[role="_abstract"]
+Enable metrics for the Kafka Bridge by setting the `bridge.metrics` configuration.
+
+.Prerequisites
+
+* xref:proc-downloading-kafka-bridge-{context}[The Kafka Bridge installation archive is downloaded].
+
+.Procedure
+
+. Set the `bridge.metrics` configuration to `strimziMetricsReporter` and related configuration.
++
+.Configuration for enabling metrics
+
+[source,properties]
+----
+bridge.metrics=strimziMetricsReporter
+----
++
+Optionally, you can set a comma-separated list of regexes used to filter exposed metrics using the `kafka.prometheus.metrics.reporter.allowlist` property.
+If not set, all available metrics will be exposed.
++
+You can add any plugin configuration to the Bridge properties file using the `kafka.` prefix.
+See the https://github.com/strimzi/metrics-reporter[Strimzi Metrics Reporter] documentation for more details.
+
+. Run the Kafka Bridge script to enable metrics.
++
+.Running the Kafka Bridge to enable metrics
+[source,shell]
+----
+./bin/kafka_bridge_run.sh --config-file=<path>/application.properties
+----
++
+With metrics enabled, you can use `GET /metrics` with the `/metrics` endpoint to retrieve Kafka Bridge metrics in Prometheus format.

pom.xml

@@ -115,6 +115,7 @@
 		<checkstyle.version>10.12.2</checkstyle.version>
 		<hamcrest.version>2.2</hamcrest.version>
 		<junit.version>5.8.2</junit.version>
+		<mockito.version>4.11.0</mockito.version>
 		<maven-jar-plugin.version>3.3.0</maven-jar-plugin.version>
 		<maven-surefire-plugin.version>3.0.0-M7</maven-surefire-plugin.version>
 		<maven-failsafe-plugin.version>3.0.0-M7</maven-failsafe-plugin.version>
@@ -131,6 +132,7 @@
 		<spotbugs.version>4.7.3</spotbugs.version>
 		<maven.spotbugs.version>4.7.3.0</maven.spotbugs.version>
 		<strimzi-oauth.version>0.15.0</strimzi-oauth.version>
+		<strimzi-metrics-reporter.version>0.1.0</strimzi-metrics-reporter.version>
 		<opentelemetry.version>1.34.1</opentelemetry.version>
 		<opentelemetry-alpha.version>1.34.1-alpha</opentelemetry-alpha.version>
 		<opentelemetry.instrumentation.version>1.32.0-alpha</opentelemetry.instrumentation.version>
@@ -295,6 +297,11 @@
 			<artifactId>micrometer-registry-prometheus</artifactId>
 			<version>${micrometer.version}</version>
 		</dependency>
+		<dependency>
+			<groupId>io.strimzi</groupId>
+			<artifactId>metrics-reporter</artifactId>
+			<version>${strimzi-metrics-reporter.version}</version>
+		</dependency>
 		<dependency>
 			<groupId>io.prometheus.jmx</groupId>
 			<artifactId>collector</artifactId>
@@ -305,6 +312,16 @@
 			<artifactId>prometheus-metrics-model</artifactId>
 			<version>${prometheus-client.version}</version>
 		</dependency>
+		<dependency>
+			<groupId>io.prometheus</groupId>
+			<artifactId>prometheus-metrics-instrumentation-jvm</artifactId>
+			<version>${prometheus-client.version}</version>
+		</dependency>
+		<dependency>
+			<groupId>io.prometheus</groupId>
+			<artifactId>prometheus-metrics-exporter-httpserver</artifactId>
+			<version>${prometheus-client.version}</version>
+		</dependency>
 		<dependency>
 			<groupId>io.prometheus</groupId>
 			<artifactId>prometheus-metrics-exposition-textformats</artifactId>
@@ -421,6 +438,12 @@
 			<version>${hamcrest.version}</version>
 			<scope>test</scope>
 		</dependency>
+		<dependency>
+			<groupId>org.mockito</groupId>
+			<artifactId>mockito-core</artifactId>
+			<version>${mockito.version}</version>
+			<scope>test</scope>
+		</dependency>
 		<dependency>
 			<groupId>io.strimzi</groupId>
 			<artifactId>strimzi-test-container</artifactId>
@@ -584,6 +607,10 @@
 									<!-- OpenTelemetry - used via classpath configuration opentelemetry-exporter-sender-grpc is required at runtime to replace OKHTTP -->
 									<ignoredUnusedDeclaredDependency>io.grpc:grpc-netty-shaded:jar</ignoredUnusedDeclaredDependency>
 									<ignoredUnusedDeclaredDependency>com.google.guava:guava</ignoredUnusedDeclaredDependency>
+									<!-- Strimzi Metrics Reporter - used via classpath configuration -->
+									<ignoredUnusedDeclaredDependency>io.strimzi:metrics-reporter</ignoredUnusedDeclaredDependency>
+									<ignoredUnusedDeclaredDependency>io.prometheus:prometheus-metrics-instrumentation-jvm</ignoredUnusedDeclaredDependency>
+									<ignoredUnusedDeclaredDependency>io.prometheus:prometheus-metrics-exporter-httpserver</ignoredUnusedDeclaredDependency>
 								</ignoredUnusedDeclaredDependencies>
 						</configuration>
 					</execution>