micromamba freezes on systems with high number of CPU cores

[sl077]

Troubleshooting docs

• My problem is not solved in the Troubleshooting docs

Anaconda default channels

• I do NOT use the Anaconda default channels (pkgs/* etc.)

How did you install Mamba?

Micromamba

Search tried in issue tracker

Yes

Latest version of Mamba

• My problem is not solved with the latest version

Tried in Conda?

I do not have this problem with Conda, just with Mamba

Describe your issue

I'm running docker on a server with dual AMD EPYC 9754 128-Core CPUs and a current Debian Trixie. Recently I had to install the docker-buildx package as it was a new requirement for the service building the docker images. Afterwards any docker build using micromamba froze while setting up the base environment.
At the same time there where hundreds of concurrent python processes running on the server, started by a script created by micromamba in /tmp, corresponding to this file:

mamba/libmamba/data/compile_pyc.py

Lines 1 to 26 in 899af78

	import os
	import sys
	from compileall import compile_file
	from concurrent.futures import ProcessPoolExecutor
	def main():
	max_workers = int(os.environ.get("MAMBA_EXTRACT_THREADS", "0"))
	if max_workers <= 0:
	max_workers = None
	results = []
	with sys.stdin:
	with ProcessPoolExecutor(max_workers=max_workers) as executor:
	while True:
	name = sys.stdin.readline().strip()
	if not name:
	break
	results.append(executor.submit(compile_file, name, quiet=1))
	success = all(r.result() for r in results)
	return success
	if __name__ == "__main__":
	success = main()
	sys.exit(int(not success))

As MAMBA_EXTRACT_THREADS isn't usually set, the number of workers this code creates equals the number of logical CPU cores found in the system, in my case 512.

My build froze after having started 506 workers and I think that is because each worker increases the number of open files, which by then reached 1024 for the main process. That wasn't a problem with the classic "docker build", as it was setting the nofile limit to 1073741816, but "docker buildx" uses the system default of 1024. Of course you can set MAMBA_EXTRACT_THREADS or add "--ulimit nofile=65536:65536" to the docker build and it will work again, but I think that micromamba should limit max_workers by default, so that it works with the default nofiles settings.

To reproduce this issue, you can use any CPU, just set MAMBA_EXTRACT_THREADS=512 and run micromamba, as this is enough to trigger the problem.

mamba info / micromamba info

libmamba version : 2.3.2
     micromamba version : 2.3.2
           curl version : libcurl/8.14.1 OpenSSL/3.5.2 zlib/1.3.1 zstd/1.5.7 libssh2/1.11.1 nghttp2/1.64.0
     libarchive version : libarchive 3.8.1 zlib/1.3.1 bz2lib/1.0.8 libzstd/1.5.7 libb2/bundled
       envs directories : /srv/conda/envs
          package cache : /srv/conda/pkgs
                          /root/.mamba/pkgs
            environment : base
           env location : /srv/conda
      user config files : /root/.mambarc
 populated config files : /srv/conda/.condarc
       virtual packages : __unix=0=0
                          __linux=6.12.43=0
                          __glibc=2.41=0
                          __archspec=1=x86_64_v3
               channels : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
       base environment : /srv/conda
               platform : linux-64

Logs

environment.yml

~/.condarc

channel_priority: strict
channels:
  - conda-forge