Open
Description
Hi,
I have had a few issues with trying to create and build neurodocker container with AFNI. I'm new to neurodocker, so I don't know if maybe I'm doing something wrong and what is or isn't relevant, so I will try to be concise.
- I followed the protocol at https://www.repronim.org/neurodocker/user_guide/examples.html, and while the first method worked, the second, which also installs R packages, failed. The initial command worked:
neurodocker generate docker \
--pkg-manager yum \
--base-image fedora:40 \
--afni method=binaries version=latest install_r_pkgs=true \
> afni-binaries-r.Dockerfile
but the build failed after about 40 mins of building (I had to cut some of the output for size limitations):
docker build --tag afni:latest-with-r --file afni-binaries-r.Dockerfile .
[+] Building 2816.3s (6/7) docker:desktop-linux
=> [internal] load build definition from afni-binaries-r.Dockerfile 0.0s
=> => transferring dockerfile: 5.83kB 0.0s
=> [internal] load metadata for docker.io/library/fedora:40 1.3s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [1/4] FROM docker.io/library/fedora:40@sha256:3c86d25fef9d2001712bc3d9b091fc40cf04be4767e48f1aa3b785bf58d300ed 0.0s
=> => resolve docker.io/library/fedora:40@sha256:3c86d25fef9d2001712bc3d9b091fc40cf04be4767e48f1aa3b785bf58d300ed 0.0s
=> CACHED [2/4] RUN export ND_ENTRYPOINT="/neurodocker/startup.sh" && if ls /etc/yum.repos.d/CentOS-* >/dev/null 2>&1; then sed -i 's/mirr 0.0s
=> ERROR [3/4] RUN yum install -y -q R cmake curl ed gsl libGLU libXp 2814.9s
------
> [3/4] RUN yum install -y -q R cmake curl ed gsl libGLU libXp libXpm libcurl-devel libgomp libjpeg-turbo-devel libpng12 mesa-dri-drivers mesa-dri-drivers mesa-libGLw ncurses-compat-libs netpbm-progs openmotif openssl-devel python-is-python3 python3-pip tcsh udunits2-devel unzip wget which which xorg-x11-fonts-misc xorg-x11-server-Xvfb && yum clean all && rm -rf /var/cache/yum/* && gsl_path="$(find / -name 'libgsl.so.??' || printf '')" && if [ -n "$gsl_path" ]; then ln -sfv "$gsl_path" "$(dirname $gsl_path)/libgsl.so.0"; fi && ldconfig && mkdir -p /opt/afni-latest && echo "Downloading AFNI ..." && curl -fL https://afni.nimh.nih.gov/pub/dist/tgz/linux_openmp_64.tgz | tar -xz -C /opt/afni-latest --strip-components 1 && yum install -y -q R-devel && yum clean all && rm -rf /var/cache/yum/* && rPkgsInstall -pkgs ALL:
528.2
528.2 Upgraded:
528.2 elfutils-libelf-0.192-9.fc40.aarch64 elfutils-libs-0.192-9.fc40.aarch64
528.2 gnutls-3.8.9-1.fc40.aarch64 libxcrypt-4.4.38-7.fc40.aarch64
528.2 openssl-libs-1:3.2.4-1.fc40.aarch64 python3-3.12.9-3.fc40.aarch64
528.2 python3-libs-3.12.9-3.fc40.aarch64
528.2 Installed:
528.2 ModemManager-glib-1.22.0-3.fc40.aarch64
528.2 NetworkManager-libnm-1:1.46.6-1.fc40.aarch64
528.2 R-4.4.3-1.fc40.aarch64
528.2 R-core-4.4.3-1.fc40.aarch64
528.2 R-core-devel-4.4.3-1.fc40.aarch64
528.2 R-devel-4.4.3-1.fc40.aarch64
528.2 R-java-4.4.3-1.fc40.aarch64
528.2 R-java-devel-4.4.3-1.fc40.aarch64
528.2 R-rpm-macros-1.2.1-10.fc40.noarch
528.2 abattis-cantarell-vf-fonts-0.301-12.fc40.noarch
...
528.3
528.4 17 files removed
528.5 '/usr/lib64/libgsl.so.0' -> '/usr/lib64/libgsl.so.27'
528.5 Downloading AFNI ...
528.5 % Total % Received % Xferd Average Speed Time Time Time Current
528.5 Dload Upload Total Spent Left Speed
100 1080M 100 1080M 0 0 487k 0 0:37:51 0:37:51 --:--:-- 649k
2814.7 17 files removed
2814.8
2814.8 oo Warning:
2814.8 Failed to load R_io.so with this error message:
2814.8 /opt/afni-latest/R_io.so
2814.8 Error in dyn.load(ll) :
2814.8 unable to load shared object '/opt/afni-latest/R_io.so':
2814.8 /opt/afni-latest/R_io.so: cannot open shared object file: No such file or directory
2814.8 Calls: source ... withVisible -> eval -> eval -> set_R_io -> dyn.load
2814.8 Execution halted
------
afni-binaries-r.Dockerfile:32
--------------------
31 | AFNI_PLUGINPATH="/opt/afni-latest"
32 | >>> RUN yum install -y -q \
33 | >>> R \
34 | >>> cmake \
35 | >>> curl \
36 | >>> ed \
37 | >>> gsl \
38 | >>> libGLU \
39 | >>> libXp \
40 | >>> libXpm \
41 | >>> libcurl-devel \
42 | >>> libgomp \
43 | >>> libjpeg-turbo-devel \
44 | >>> libpng12 \
45 | >>> mesa-dri-drivers \
46 | >>> mesa-dri-drivers \
47 | >>> mesa-libGLw \
48 | >>> ncurses-compat-libs \
49 | >>> netpbm-progs \
50 | >>> openmotif \
51 | >>> openssl-devel \
52 | >>> python-is-python3 \
53 | >>> python3-pip \
54 | >>> tcsh \
55 | >>> udunits2-devel \
56 | >>> unzip \
57 | >>> wget \
58 | >>> which \
59 | >>> which \
60 | >>> xorg-x11-fonts-misc \
61 | >>> xorg-x11-server-Xvfb \
62 | >>> && yum clean all \
63 | >>> && rm -rf /var/cache/yum/* \
64 | >>> && gsl_path="$(find / -name 'libgsl.so.??' || printf '')" \
65 | >>> && if [ -n "$gsl_path" ]; then \
66 | >>> ln -sfv "$gsl_path" "$(dirname $gsl_path)/libgsl.so.0"; \
67 | >>> fi \
68 | >>> && ldconfig \
69 | >>> && mkdir -p /opt/afni-latest \
70 | >>> && echo "Downloading AFNI ..." \
71 | >>> && curl -fL https://afni.nimh.nih.gov/pub/dist/tgz/linux_openmp_64.tgz \
72 | >>> | tar -xz -C /opt/afni-latest --strip-components 1 \
73 | >>> && yum install -y -q \
74 | >>> R-devel \
75 | >>> && yum clean all \
76 | >>> && rm -rf /var/cache/yum/* \
77 | >>> && rPkgsInstall -pkgs ALL
78 | ENTRYPOINT ["/neurodocker/startup.sh"]
--------------------
ERROR: failed to solve: process "/bin/sh -c yum install -y -q R cmake curl ed gsl libGLU libXp libXpm libcurl-devel libgomp libjpeg-turbo-devel libpng12 mesa-dri-drivers mesa-dri-drivers mesa-libGLw ncurses-compat-libs netpbm-progs openmotif openssl-devel python-is-python3 python3-pip tcsh udunits2-devel unzip wget which which xorg-x11-fonts-misc xorg-x11-server-Xvfb && yum clean all && rm -rf /var/cache/yum/* && gsl_path=\"$(find / -name 'libgsl.so.??' || printf '')\" && if [ -n \"$gsl_path\" ]; then ln -sfv \"$gsl_path\" \"$(dirname $gsl_path)/libgsl.so.0\"; fi && ldconfig && mkdir -p /opt/afni-latest && echo \"Downloading AFNI ...\" && curl -fL https://afni.nimh.nih.gov/pub/dist/tgz/linux_openmp_64.tgz | tar -xz -C /opt/afni-latest --strip-components 1 && yum install -y -q R-devel && yum clean all && rm -rf /var/cache/yum/* && rPkgsInstall -pkgs ALL" did not complete successfully: exit code: 1
- I was unable to run neurodocker with specifying AFNI version, e.g.:
neurodocker generate docker \
--pkg-manager yum \
--base-image fedora:40 \
--afni method=binaries version=25.1.03 \
> version.Dockerfile
Traceback (most recent call last):
File "/opt/anaconda3/lib/python3.11/site-packages/neurodocker/reproenv/renderers.py", line 76, in _render_string_from_template
source = tmpl.render(template=template)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/lib/python3.11/site-packages/jinja2/environment.py", line 1301, in render
self.environment.handle_exception()
File "/opt/anaconda3/lib/python3.11/site-packages/jinja2/environment.py", line 936, in handle_exception
raise rewrite_traceback_stack(source=source)
File "<template>", line 13, in top-level template code
jinja2.exceptions.UndefinedError: 'dict object' has no attribute '25.1.03'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/anaconda3/bin/neurodocker", line 8, in <module>
sys.exit(cli())
^^^^^
File "/opt/anaconda3/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/lib/python3.11/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/opt/anaconda3/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/lib/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/lib/python3.11/site-packages/neurodocker/cli/generate.py", line 478, in docker
_base_generate(
File "/opt/anaconda3/lib/python3.11/site-packages/neurodocker/cli/generate.py", line 455, in _base_generate
r = renderer.from_dict(renderer_dict)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/lib/python3.11/site-packages/neurodocker/reproenv/renderers.py", line 212, in from_dict
renderer.add_registered_template(method_or_template, **kwds)
File "/opt/anaconda3/lib/python3.11/site-packages/neurodocker/reproenv/renderers.py", line 364, in add_registered_template
self.add_template(template=template, method=method)
File "/opt/anaconda3/lib/python3.11/site-packages/neurodocker/reproenv/renderers.py", line 328, in add_template
command = _render_string_from_template(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/lib/python3.11/site-packages/neurodocker/reproenv/renderers.py", line 78, in _render_string_from_template
raise RendererError(err) from e
neurodocker.reproenv.exceptions.RendererError: A template included in this renderer raised an error. Please check the template definition. A required argument might not be included in the required arguments part of the template. Variables in the template should start with `self.`.
- Maybe similar to this issue, AFNI out of date in distributed version, and maybe its build is broken? #662. I first tried building it with Ubuntu 22.04 and 24.04. The first issue is that it fails on
multiarch-support, so the file generation is fine:
neurodocker generate docker \
--pkg-manager apt \
--base-image ubuntu:22.04 \
--afni version=latest \
--miniconda version=latest \
env_name=neuro \
conda_install='python=3.11 nipype nilearn pandas matplotlib' \
--user=neuro \
> Dockerfile_neurodocker
but the build failed on:
76 | >>> | tar -xz -C /opt/afni-latest --strip-components 1
77 | ENV CONDA_DIR="/opt/miniconda-latest" \
--------------------
ERROR: failed to solve: process "/bin/sh -c apt-get update -qq && apt-get install -y -q --no-install-recommends ca-certificates cmake curl ed gsl-bin libcurl4-openssl-dev libgl1-mesa-dri libglib2.0-0 libglu1-mesa-dev libglw1-mesa libgomp1 libjpeg-turbo8-dev libjpeg62 libssl-dev libudunits2-dev libxm4 multiarch-support netpbm python-is-python3 python3-pip tcsh xfonts-base xvfb && rm -rf /var/lib/apt/lists/* && _reproenv_tmppath=\"$(mktemp -t tmp.XXXXXXXXXX.deb)\" && curl -fsSL --retry 5 -o \"${_reproenv_tmppath}\" http://mirrors.kernel.org/debian/pool/main/libx/libxp/libxp6_1.0.2-2_amd64.deb && apt-get install --yes -q \"${_reproenv_tmppath}\" && rm \"${_reproenv_tmppath}\" && _reproenv_tmppath=\"$(mktemp -t tmp.XXXXXXXXXX.deb)\" && curl -fsSL --retry 5 -o \"${_reproenv_tmppath}\" http://snapshot.debian.org/archive/debian-security/20160113T213056Z/pool/updates/main/libp/libpng/libpng12-0_1.2.49-1%2Bdeb7u2_amd64.deb && apt-get install --yes -q \"${_reproenv_tmppath}\" && rm \"${_reproenv_tmppath}\" && apt-get update -qq && apt-get install --yes --quiet --fix-missing && rm -rf /var/lib/apt/lists/* && gsl_path=\"$(find / -name 'libgsl.so.??' || printf '')\" && if [ -n \"$gsl_path\" ]; then ln -sfv \"$gsl_path\" \"$(dirname $gsl_path)/libgsl.so.0\"; fi && ldconfig && mkdir -p /opt/afni-latest && echo \"Downloading AFNI ...\" && curl -fL https://afni.nimh.nih.gov/pub/dist/tgz/linux_openmp_64.tgz | tar -xz -C /opt/afni-latest --strip-components 1" did not complete successfully: exit code: 100
This can be fixed by removing the multiarch-support flag from the build file but when you do that, it fails on yet another thing - curl -fL https://afni.nimh.nih.gov/pub/dist/tgz/linux_openmp_64.tgz which is the reason it might be related to the other issue I mentioned.
82 | >>> && rPkgsInstall -pkgs ALL
83 | ENV CONDA_DIR="/opt/miniconda-latest" \
--------------------
ERROR: failed to solve: process "/bin/sh -c apt-get update -qq && apt-get install -y -q --no-install-recommends ca-certificates cmake curl ed gsl-bin libcurl4-openssl-dev libgl1-mesa-dri libglib2.0-0 libglu1-mesa-dev libglw1-mesa libgomp1 libjpeg-turbo8-dev libjpeg62 libssl-dev libudunits2-dev libxm4 netpbm python-is-python3 python3-pip tcsh xfonts-base xvfb && rm -rf /var/lib/apt/lists/* && _reproenv_tmppath=\"$(mktemp -t tmp.XXXXXXXXXX.deb)\" && curl -fsSL --retry 5 -o \"${_reproenv_tmppath}\" http://mirrors.kernel.org/debian/pool/main/libx/libxp/libxp6_1.0.2-2_amd64.deb && apt-get install --yes -q \"${_reproenv_tmppath}\" && rm \"${_reproenv_tmppath}\" && _reproenv_tmppath=\"$(mktemp -t tmp.XXXXXXXXXX.deb)\" && curl -fsSL --retry 5 -o \"${_reproenv_tmppath}\" http://snapshot.debian.org/archive/debian-security/20160113T213056Z/pool/updates/main/libp/libpng/libpng12-0_1.2.49-1%2Bdeb7u2_amd64.deb && apt-get install --yes -q \"${_reproenv_tmppath}\" && rm \"${_reproenv_tmppath}\" && apt-get update -qq && apt-get install --yes --quiet --fix-missing && rm -rf /var/lib/apt/lists/* && gsl_path=\"$(find / -name 'libgsl.so.??' || printf '')\" && if [ -n \"$gsl_path\" ]; then ln -sfv \"$gsl_path\" \"$(dirname $gsl_path)/libgsl.so.0\"; fi && ldconfig && mkdir -p /opt/afni-latest && echo \"Downloading AFNI ...\" && curl -fL https://afni.nimh.nih.gov/pub/dist/tgz/linux_openmp_64.tgz | tar -xz -C /opt/afni-latest --strip-components 1 && apt-get update -qq && apt-get install -y -q --no-install-recommends libnlopt-dev r-base r-base-dev && rm -rf /var/lib/apt/lists/* && rPkgsInstall -pkgs ALL" did not complete successfully: exit code: 22
I didn't manage to get any further than that, and I'd welcome any tips and advice on how to proceed.
Thanks for helping.