2022-11-07 jupyterlab surface topography docker recipe

[jotelha]

This is basically a recipe building on the https://jupyter-docker-stacks.readthedocs.io/en/latest/ jupyterlab docker stack, adding a ipython kernel that "breaks out" of the default conda environment within these containers and installs a simple list of packages from requirements.txt. The SurfaceTopography package (https://github.com/ContactEngineering/SurfaceTopography) needs some special treatment, without it the recipe coul look much simpler.

I launch such images on bwCloud, the README.md file documents to a certain extent how.
Hope that works for you !

[jotelha]

@sannant do you want to modify the packages in docker/jupyterlab/jupyterlab-SurfaceTopography/requirements.in a little and put in whatever a public image should have? I'd then be happy to build and push this to docker hub.

[sannant]

Yes, you mean adding all the ContactMechanics and Adhesion stuff I want ?
And eventual other useful libraries ?

[sannant]

Since ContactMechanics needs to be installed after SurfaceTopography, I should also modify the DockerFile, right ?

[jotelha]

Since ContactMechanics needs to be installed after SurfaceTopography, I should also modify the DockerFile, right ?

Right now, the requirements.txt is actually processed after installing SurfaceTopography, see Dockerfile from line 34,

# Workaround for building packages and introducing an ipykernel outside of the jupyterlab conda environment
RUN PATH=$(echo "$PATH" | sed -e 's|/opt/conda[^:]*:||g') && ( \
        pip install --no-binary numpy SurfaceTopography && \
        pip install ipykernel && \
        python3 -m ipykernel install --user --name SurfaceTopography --display-name "Python (SurfaceTopography)" && \
        echo "c.MultiKernelManager.default_kernel_name = 'SurfaceTopography'" >> ${HOME}/.jupyter/jupyter_notebook_config.py) && \
    fix-permissions "${CONDA_DIR}" && \
    fix-permissions "/home/${NB_USER}"

# Workaround for modifying system python outside of the jupyterlab conda environment
COPY --chown=${NB_UID}:${NB_GID} jupyterlab-SurfaceTopography/requirements.txt "/home/${NB_USER}/"
RUN PATH="/home/${NB_USER}/.local/bin:$(echo "$PATH" | sed -e 's|/opt/conda[^:]*:||g')" && ( \
        pip install --quiet --no-cache-dir --requirement "/home/${NB_USER}/requirements.txt" )

so let's just stick it into the requirements.txt and see whether that works. The only reason why SurfaceTopography is treated explicitly in the Dockerfile is because it needs that --no-binary numpy option (ContactEngineering/SurfaceTopography#239 (comment))

[jotelha]

@sannant, contactmechanics depends on mpi4py and an MPI installation, is that avoidable?

x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/usr/include/python3.8 -c _configtest.c -o _configtest.o
  _configtest.c:2:10: fatal error: mpi.h: No such file or directory
      2 | #include <mpi.h>
        |          ^~~~~~~
  compilation terminated.
  failure.
  removing: _configtest.c _configtest.o
  error: Cannot compile MPI programs. Check your configuration!!!
  ----------------------------------------
  ERROR: Failed building wheel for mpi4py
ERROR: Could not build wheels for mpi4py which use PEP 517 and cannot be installed directly

[sannant]

It shouldn't depend on that

[sannant]

https://github.com/ContactEngineering/ContactMechanics/blob/master/requirements.txt

https://github.com/ContactEngineering/ContactMechanics/blob/master/setup.py

And see how we test and build here.

python3 -m pip install -r requirements.txt --no-binary setuptools_scm --no-binary numpy

[jotelha]

Via

mpi4py==3.1.4
    # via mufft

[sannant]

Well you already install muFFT in SurfaceTopography, I do not understand

[jotelha]

Hm then I also don't understand anymore.

[jotelha]

I understand now. Because I pin the versions locally (and I have MPI installed locally), muFFT automatically inserts that mpi4py dependency!

[jotelha]

Yes, you mean adding all the ContactMechanics and Adhesion stuff I want ? And eventual other useful libraries ?

Sorry, didn't reply to this one yet. Yes, please, as long as it doesn't blow up the container to 20 GB.

And my apologies for blaming an mpi dependency on ContactMechanics, that was due to muFFT adding the dependency automatically when detecting an MPI installation in the environment, https://gitlab.com/muspectre/muspectre/-/blob/master/setup.py#L672