You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Description
I've built a Spring Cloud Config Server integrated with Spring Cloud Bus via Kafka for dynamic property refreshing.
I've another application that consumes these properties and refreshes them dynamically.
Endpoint /monitor in Spring Cloud Config Server is used to update the service properties.
The service receives RefreshRemoteApplicationEvent in springCloudBus kafka topic.
After several properties updates, we started getting the error java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
I noticed that a lot of kafka-producer-network-thread threads remain open and their number grows after each update of the properties.
I see this on both an Spring Cloud Config Server and a configurable service.
I suspect this is due to improper closing of Kafka connections after getting RefreshRemoteApplicationEvent 24c1abe#diff-5e17c4149c35b951bd7e88675c6fab6589366819948092156fd0b70573f25632R370
During connection cache cleanup, the connection is not closed beforehand, and kafka-producer-network-thread threads continue to exist in the OS.
Version of the framework org.springframework.boot:spring-boot-starter-parent:3.3.8 spring.cloud.dependencies 2023.0.5 -> org.springframework.cloud:spring-cloud-stream:jar:4.1.5
To Reproduce
Send several /monitor request to Spring Cloud Config Server or /busrefresh request to configurable service.
To view the threads, use the command jcmd <PID> Thread.print | grep -E '"main"|"RMI"|"GC"|prio=' --color=auto
You can observe an increasing number of kafka-producer-network-thread threads.
Possible solution
Close kafka connections before cache cleanup.
The text was updated successfully, but these errors were encountered:
Description
I've built a Spring Cloud Config Server integrated with Spring Cloud Bus via Kafka for dynamic property refreshing.
I've another application that consumes these properties and refreshes them dynamically.
Endpoint /monitor in Spring Cloud Config Server is used to update the service properties.
The service receives RefreshRemoteApplicationEvent in springCloudBus kafka topic.
After several properties updates, we started getting the error
java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
I noticed that a lot of kafka-producer-network-thread threads remain open and their number grows after each update of the properties.
I see this on both an Spring Cloud Config Server and a configurable service.
I suspect this is due to improper closing of Kafka connections after getting RefreshRemoteApplicationEvent
24c1abe#diff-5e17c4149c35b951bd7e88675c6fab6589366819948092156fd0b70573f25632R370
During connection cache cleanup, the connection is not closed beforehand, and kafka-producer-network-thread threads continue to exist in the OS.
Version of the framework
org.springframework.boot:spring-boot-starter-parent:3.3.8
spring.cloud.dependencies 2023.0.5
->org.springframework.cloud:spring-cloud-stream:jar:4.1.5
To Reproduce
Send several /monitor request to Spring Cloud Config Server or /busrefresh request to configurable service.
To view the threads, use the command
jcmd <PID> Thread.print | grep -E '"main"|"RMI"|"GC"|prio=' --color=auto
You can observe an increasing number of kafka-producer-network-thread threads.
Possible solution
Close kafka connections before cache cleanup.
The text was updated successfully, but these errors were encountered: