AFNI recipe does not work correctly in Singularity build

[stebo85]

This is a left-over from issue #584, but I opened a new issue to keep it focussed on the actual problem.

Building afni with singularity currently doesn't work (but works with docker):

singularity run docker://repronim/neurodocker:latest generate singularity \
    --pkg-manager yum \
    --base-image fedora:36 \
    --afni method=binaries version=latest install_r_pkgs=true > /cubric/collab/359_curiosity/tamas/misc/afni-binaries-r.Singularityfile

singularity build --remote /cubric/collab/359_curiosity/tamas/images/afni-binaries-r.sif /cubric/collab/359_curiosity/tamas/misc/afni-binaries-r.Singularityfile

This results in:

/.post.script: line 47: rPkgsInstall: command not found

@kaczmarj, Is there a difference in how Singularity files are written in Neurodocker as compared to how docker files are written that would explain why a tool is not on the path in Singularity, but is on Path when building in Docker?

Thank you
Steffen

[kaczmarj]

it seems that in Singularity, the environment variables made in %environment are not available in %post. am i correct?

i tested this using a small Singularity file

# Test whether updated environment variables like PATH are available in %post.

Bootstrap: docker
From: fedora:36

%environment
export PATH="/opt/afni-latest:$PATH"
export NEWVAR="ThisIsATest"

%post
echo $PATH
echo $NEWVAR

and during the build, I get the following output, which shows that the updated PATH variable is not available in %post.

INFO:    Starting build...
Getting image source signatures
Copying blob c7bef7d09442 skipped: already exists  
Copying config 4d934e266f done  
Writing manifest to image destination
Storing signatures
2023/10/25 10:38:22  info unpack layer: sha256:c7bef7d09442a4c4deef8e8120027e7fa3f163c7b5a0afedc3f2c4c2e7652e1c
INFO:    Running post scriptlet
+ echo /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ echo

INFO:    Adding environment to container
INFO:    Creating SIF file...
INFO:    Build complete: foobar

for help in debugging, here are the Dockerfile and Singularity file for the command

singularity run docker://repronim/neurodocker:latest generate singularity \
    --pkg-manager yum \
    --base-image fedora:36 \
    --afni method=binaries version=latest install_r_pkgs=true

Dockerfile

# Generated by Neurodocker and Reproenv.

FROM fedora:36
ENV PATH="/opt/afni-latest:$PATH" \
    AFNI_PLUGINPATH="/opt/afni-latest"
RUN yum install -y -q \
           R \
           cmake \
           curl \
           ed \
           gsl \
           libGLU \
           libXp \
           libXpm \
           libcurl-devel \
           libgomp \
           libjpeg-turbo-devel \
           libpng12 \
           mesa-dri-drivers \
           mesa-dri-drivers \
           mesa-libGLw \
           ncurses-compat-libs \
           netpbm-progs \
           openmotif \
           openssl-devel \
           python-is-python3 \
           python3-pip \
           tcsh \
           udunits2-devel \
           unzip \
           wget \
           which \
           which \
           xorg-x11-fonts-misc \
           xorg-x11-server-Xvfb \
    && yum clean all \
    && rm -rf /var/cache/yum/* \
    && gsl_path="$(find / -name 'libgsl.so.??' || printf '')" \
    && if [ -n "$gsl_path" ]; then \
         ln -sfv "$gsl_path" "$(dirname $gsl_path)/libgsl.so.0"; \
    fi \
    && ldconfig \
    && mkdir -p /opt/afni-latest \
    && echo "Downloading AFNI ..." \
    && curl -fL https://afni.nimh.nih.gov/pub/dist/tgz/linux_openmp_64.tgz \
    | tar -xz -C /opt/afni-latest --strip-components 1 \
    &&   yum install -y -q \
           R-devel \
    && yum clean all \
    && rm -rf /var/cache/yum/* \
    &&   rPkgsInstall -pkgs ALL

# Save specification to JSON.
RUN printf '{ \
  "pkg_manager": "yum", \
  "existing_users": [ \
    "root" \
  ], \
  "instructions": [ \
    { \
      "name": "from_", \
      "kwds": { \
        "base_image": "fedora:36" \
      } \
    }, \
    { \
      "name": "env", \
      "kwds": { \
        "PATH": "/opt/afni-latest:$PATH", \
        "AFNI_PLUGINPATH": "/opt/afni-latest" \
      } \
    }, \
    { \
      "name": "run", \
      "kwds": { \
        "command": "yum install -y -q \\\\\\n    R \\\\\\n    cmake \\\\\\n    curl \\\\\\n    ed \\\\\\n    gsl \\\\\\n    libGLU \\\\\\n    libXp \\\\\\n    libXpm \\\\\\n    libcurl-devel \\\\\\n    libgomp \\\\\\n    libjpeg-turbo-devel \\\\\\n    libpng12 \\\\\\n    mesa-dri-drivers \\\\\\n    mesa-dri-drivers \\\\\\n    mesa-libGLw \\\\\\n    ncurses-compat-libs \\\\\\n    netpbm-progs \\\\\\n    openmotif \\\\\\n    openssl-devel \\\\\\n    python-is-python3 \\\\\\n    python3-pip \\\\\\n    tcsh \\\\\\n    udunits2-devel \\\\\\n    unzip \\\\\\n    wget \\\\\\n    which \\\\\\n    which \\\\\\n    xorg-x11-fonts-misc \\\\\\n    xorg-x11-server-Xvfb\\nyum clean all\\nrm -rf /var/cache/yum/*\\n\\ngsl_path=\\"$\(find / -name '"'"'libgsl.so.??'"'"' || printf '"'"''"'"'\)\\"\\nif [ -n \\"$gsl_path\\" ]; then \\\\\\n  ln -sfv \\"$gsl_path\\" \\"$\(dirname $gsl_path\)/libgsl.so.0\\"; \\\\\\nfi\\nldconfig\\nmkdir -p /opt/afni-latest\\necho \\"Downloading AFNI ...\\"\\ncurl -fL https://afni.nimh.nih.gov/pub/dist/tgz/linux_openmp_64.tgz \\\\\\n| tar -xz -C /opt/afni-latest --strip-components 1\\n  yum install -y -q \\\\\\n    R-devel\\nyum clean all\\nrm -rf /var/cache/yum/*\\n  rPkgsInstall -pkgs ALL" \
      } \
    } \
  ] \
}' > /.reproenv.json
# End saving to specification to JSON.

Singularity

# Generated by Neurodocker and Reproenv.

Bootstrap: docker
From: fedora:36

%environment
export PATH="/opt/afni-latest:$PATH"
export AFNI_PLUGINPATH="/opt/afni-latest"

%post
yum install -y -q \
    R \
    cmake \
    curl \
    ed \
    gsl \
    libGLU \
    libXp \
    libXpm \
    libcurl-devel \
    libgomp \
    libjpeg-turbo-devel \
    libpng12 \
    mesa-dri-drivers \
    mesa-dri-drivers \
    mesa-libGLw \
    ncurses-compat-libs \
    netpbm-progs \
    openmotif \
    openssl-devel \
    python-is-python3 \
    python3-pip \
    tcsh \
    udunits2-devel \
    unzip \
    wget \
    which \
    which \
    xorg-x11-fonts-misc \
    xorg-x11-server-Xvfb
yum clean all
rm -rf /var/cache/yum/*

gsl_path="$(find / -name 'libgsl.so.??' || printf '')"
if [ -n "$gsl_path" ]; then \
  ln -sfv "$gsl_path" "$(dirname $gsl_path)/libgsl.so.0"; \
fi
ldconfig
mkdir -p /opt/afni-latest
echo "Downloading AFNI ..."
curl -fL https://afni.nimh.nih.gov/pub/dist/tgz/linux_openmp_64.tgz \
| tar -xz -C /opt/afni-latest --strip-components 1
  yum install -y -q \
    R-devel
yum clean all
rm -rf /var/cache/yum/*
  rPkgsInstall -pkgs ALL

# Save specification to JSON.
printf '{ \
  "pkg_manager": "yum", \
  "existing_users": [ \
    "root" \
  ], \
  "instructions": [ \
    { \
      "name": "from_", \
      "kwds": { \
        "base_image": "fedora:36" \
      } \
    }, \
    { \
      "name": "env", \
      "kwds": { \
        "PATH": "/opt/afni-latest:$PATH", \
        "AFNI_PLUGINPATH": "/opt/afni-latest" \
      } \
    }, \
    { \
      "name": "run", \
      "kwds": { \
        "command": "yum install -y -q \\\\\\n    R \\\\\\n    cmake \\\\\\n    curl \\\\\\n    ed \\\\\\n    gsl \\\\\\n    libGLU \\\\\\n    libXp \\\\\\n    libXpm \\\\\\n    libcurl-devel \\\\\\n    libgomp \\\\\\n    libjpeg-turbo-devel \\\\\\n    libpng12 \\\\\\n    mesa-dri-drivers \\\\\\n    mesa-dri-drivers \\\\\\n    mesa-libGLw \\\\\\n    ncurses-compat-libs \\\\\\n    netpbm-progs \\\\\\n    openmotif \\\\\\n    openssl-devel \\\\\\n    python-is-python3 \\\\\\n    python3-pip \\\\\\n    tcsh \\\\\\n    udunits2-devel \\\\\\n    unzip \\\\\\n    wget \\\\\\n    which \\\\\\n    which \\\\\\n    xorg-x11-fonts-misc \\\\\\n    xorg-x11-server-Xvfb\\nyum clean all\\nrm -rf /var/cache/yum/*\\n\\ngsl_path=\\"$\(find / -name '"'"'libgsl.so.??'"'"' || printf '"'"''"'"'\)\\"\\nif [ -n \\"$gsl_path\\" ]; then \\\\\\n  ln -sfv \\"$gsl_path\\" \\"$\(dirname $gsl_path\)/libgsl.so.0\\"; \\\\\\nfi\\nldconfig\\nmkdir -p /opt/afni-latest\\necho \\"Downloading AFNI ...\\"\\ncurl -fL https://afni.nimh.nih.gov/pub/dist/tgz/linux_openmp_64.tgz \\\\\\n| tar -xz -C /opt/afni-latest --strip-components 1\\n  yum install -y -q \\\\\\n    R-devel\\nyum clean all\\nrm -rf /var/cache/yum/*\\n  rPkgsInstall -pkgs ALL" \
      } \
    } \
  ] \
}' > /.reproenv.json
# End saving to specification to JSON.

[kaczmarj]

indeed, singularity does not make %environment variables available in %post by default...

related:

• %environment during 'singularity build' executed in host namespace instead of container namespace apptainer/singularity#1053 (comment)
• %environment variables not available in %post apptainer/singularity#1284

i think the solution would be to add . $SINGULARITY_ENVIRONMENT to the beginning of %post.

@stebo85 - what do you think?

if apptainer / singularity developers are reading, then i would like to say that i find this behavior highly unexpected.

[stebo85]

Dear @kaczmarj - great find - that explains the problems.

I tried adding . $SINGULARITY_ENVIRONMENT to the %post section, but this results in:

INFO:    Running post scriptlet
+ . /.singularity.d/env/91-environment.sh
/.post.script: line 1: /.singularity.d/env/91-environment.sh: No such file or directory
FATAL:   While performing build: while running engine: exit status 1

I know it's not elegant, but would anything else speak against re-exporting the environment variables in the post section?

Bootstrap: docker
From: fedora:36

%environment
export PATH="/opt/afni-latest:$PATH"
export NEWVAR="ThisIsATest"

%post
export PATH="/opt/afni-latest:$PATH"
export NEWVAR="ThisIsATest"

echo $PATH
echo $NEWVAR

at least it works :?

INFO:    Running post scriptlet
+ export PATH=/opt/afni-latest:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ PATH=/opt/afni-latest:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ export NEWVAR=ThisIsATest
+ NEWVAR=ThisIsATest
+ echo /opt/afni-latest:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
/opt/afni-latest:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ echo ThisIsATest
ThisIsATest

@vsoch - could you maybe have quick look at what we are getting wrong here?

Kind regards
Steffen

[kaczmarj]

I know it's not elegant, but would anything else speak against re-exporting the environment variables in the post section?

fine with me if there's no better solution.

[github-actions]

This issue is stale because it has been open for 30 days with no activity.