Excessive JDK HttpClient Instantiation by Apicurio Java Client

[foal]

Client Version: 2.6.12
Java Version: 21
Spring Boot Version: 3.4.7

Description

When using the Apicurio Java client in a Spring Boot application configured via properties, the framework leverages default constructors to instantiate AvroKafkaSerializer and AvroKafkaDeserializer.

As a result, each instance of AvroKafkaDeserializer or AvroKafkaSerializer creates a new java.net.http.HttpClient. This leads to:

• Increased thread usage: ~20 additional threads for simple consumers.
• Severe thread explosion in complex applications, e.g. thousands of threads for an app with 8 Kafka Streams applications.
• In the worst-case scenario, the JVM crashes with an "unable to create native thread" error due to thread exhaustion.

Environment

Affects any environment using Spring Boot property-based configuration with Apicurio client.

Steps to Reproduce

1. Configure a Spring Boot application using property-based configuration to use Apicurio’s AvroKafkaDeserializer.
2. Set up a Kafka topic with multiple partitions.
3. Launch the application and monitor thread usage.
4. Optional: Add multiple Kafka Streams application to observe larger impact.

Expected Behavior

A single HttpClient instance should be shared across all (de)serializer instances, ensuring resource efficiency and thread safety.

Actual Behavior

A new HttpClient is instantiated for every AvroKafkaDeserializer or AvroKafkaSerializer instance, causing uncontrolled thread growth.

Impact

• Degrades performance and stability in production environments.
• Can cause JVM crashes due to native thread limits being exceeded.

Proposed Solution

Introduce a shared, lazily initialized static HttpClient that can be reused across instances.