Malloc tuning rationale

[jakubmalek]

Hi,
recently I've discovered that the JVM build-pack is tuning the malloc with fixed number of arenas set to value of 2.
https://github.com/paketo-buildpacks/libjvm/blob/main/jre.go#L143

I'm generally aware that using the default, which is based on number of CPUs can lead to high memory fragmentation. I've discovered the same problem in my other JVM container, not using the build packs, so I know this can be indeed a problem.
I did find some blogs, describing the problem and inner working of glibc malloc implementation:

• https://blog.malt.engineering/java-in-k8s-how-weve-reduced-memory-usage-without-changing-any-code-cbef5d740ad
• https://sploitfun.wordpress.com/2015/02/10/understanding-glibc-malloc/
• Consider lowering MALLOC_ARENA_MAX to prevent native memory OOM prestodb/presto#8993

But the thing, that is somehow missing for me, is why exactly number 2 was selected as the default ?

• Are there any special reasons for the JVM, justifying the use of 2 malloc arenas ?
• Are there any concerns in regards to performance with lower number of arenas ?

I hope I didn't miss some obvious link to the discussion or documentation, which explain it.

[dmikusa]

recently I've discovered that the JVM build-pack is tuning the malloc with fixed number of arenas set to value of 2.

Correct & I can confirm that is an intentional setting.

I'm generally aware that using the default, which is based on number of CPUs can lead to high memory fragmentation.

My understanding as well.

But the thing, that is somehow missing for me, is why exactly number 2 was selected as the default ?

My understanding is that this predates Paketo buildpacks and goes back to CloudFoundry buildpacks. The Java buildpack there also sets the max arenas too.

cloudfoundry/java-buildpack@d1a21ff

The rationale is described in that commit and the linked issue. I believe the same should apply for Paketo Java buildpacks.

Are there any special reasons for the JVM, justifying the use of 2 malloc arenas ?

I think the idea is that the JVM doesn't know about or take into consideration what malloc is doing, primarily because it runs on many different operating systems which may do things differently. Thus you may get into situations where the JVM is causing memory arenas to be used in a suboptimal way. I don't know if that is the truth or how likely you're to see that in practice, but that's the idea behind it.

Are there any concerns in regards to performance with lower number of arenas ?

My understanding is that if you have the JVM set to consume all your heap up front, probably not. It's going to allocate that memory with the OS when your app starts so there's very little chance it'll cause issues.

As with most things, it likely depends (i.e. your mileage will vary). It's technically not the optimal way to do things so if you have a highly performance-critical application, you should test to verify.