Adapting trame/paraview/Docker example so that it can support multiple-users

[smkacur]

Hi, I would like to take the paraview/docker deployment example shown here and adapt it so that it has multi-users enabled like in the trame coockie-cutter example. Unlike, the cookie cutter, however, I would like to stick with the single file application format.

So far I have modified the example by adding a /docker directory which contains the scripts and setup directories and also the server directory after the build_image.sh script is executed.

My first question is, since the single file application is not installable, should I still be mounting the ROOT_DIR in the build_server.sh script?

The second question pertains to the dockerfile. In the current paraview/docker example, the entire directory containing the dockerfile and the SimpleCone.py application gets copied into the image under the /deploy directory. But when enabling multipe-users, the ./server directory gets copied into /deploy/server. So, since the single file application is not installable and doesn't get installed under the deploy directory, where should the SimpleCone.py file be copied into the image?

Thank you!

[jourdain]

Couple of question and validation question.

1. Your trame app is a single file and therefore, you don't want to have a python package for it like the cookie cutter is forcing you to do? [True/False]
2. You want to split the build step in two rather than using a single docker build step like described in the example? [True/False]

[smkacur]

1. True
2. Are you referring to the single file paraview/Docker example? I have split the build in two (following the cookie cutter example) to first build the server directory which can then be copied into the image.

[jourdain]

For the 2, I think staying with a single step doing RUN /opt/trame/entrypoint.sh build is simpler. The 2 steps are nice to debug and see what got built, but when you have a single file, it might be overkill.

[smkacur]

Thank you for that advice, it is much simpler. For my own understanding, does the simple paraview/Docker example I referenced above enable multi-user? I thought that it did not because when checking the logs under /deploy/sever/logs/launcher in the container I see this message:

ParaView is using venv: /deploy/server/venv

App running at:
 - Local:   http://0.0.0.0:9498/
 - Network: http://0.0.0.0:9498/

Note that for multi-users you need to use and configure a launcher.

But the RUN /opt/trame/entrypoint.sh build builds the server directory and configures the launcher. Also I can open the application in two separate browsers (chome + safari) and manipulate them independently of each other.

[jourdain]

The trame process in itself is not aware of its surrounding (docker). But the docker setup is only meant to manage the multi-users case.

[smkacur]

Following up on this. The simplified deployment works with the kitware/trame image I posted above (which builds paraview within the image) but I am running into a new problem when I try to deploy with a modified image.

We are building the kitware/trame image using our own paraview build (as paraview is also used in other parts of our stack). And when I try to deploy the application I am running into this error: /opt/paraview/bin/pvpython: error while loading shared libraries: libpython3.10.so.1.0: cannot open shared object file: No such file or directory.

Initially, I thought it may have been caused by trying to build the kitware/trame:py3.10-ubuntu22.04 image on a mac (arm) but I get the same error when I build the images with amd64 as well. Do you have any advice?

Here is the kitware/trame image we are trying to use:

FROM kitware/trame:py3.10-ubuntu22.04 as paraview-trame
RUN apt-get update && apt-get install -y \
    libx11-dev \
    libosmesa6-dev \
    libhdf5-dev \
    libgdal-dev \
    ffmpeg \
    libglvnd-core-dev \
  && rm -rf /var/lib/apt/lists
COPY --from=paraview-build /opt/ssi/paraview-build/bin /opt/paraview/bin
COPY --from=paraview-build /opt/ssi/paraview-build/lib /opt/paraview/lib
COPY --from=paraview-build /opt/ssi/paraview-build/share /opt/paraview/share

Here is the build_paraview.sh that we are using to build paraview:

#!/bin/bash
if [ -z "$1" ] 
then
  VER=release
else
  VER=$1
fi

mkdir paraview-build
git clone https://gitlab.kitware.com/paraview/paraview.git
cd paraview
git checkout $VER
git submodule update --init --recursive
cd ../paraview-build
cmake -DPARAVIEW_USE_PYTHON=ON \
      -DPARAVIEW_ENABLE_GDAL=ON \
      -DPARAVIEW_ENABLE_FFMPEG=ON \
      -DVTKm_ENABLE_HDF5_IO=ON  \
      -DPARAVIEW_USE_QT=OFF \
      -DPARAVIEW_ENABLE_XDMF2=ON \
      -DPARAVIEW_ENABLE_XDMF3=ON \
      -DPARAVIEW_ENABLE_VISITBRIDGE=ON \
      -DVTK_USE_X=OFF \
      -DVTK_OPENGL_HAS_OSMESA=ON \
      -DCMAKE_BUILD_TYPE=Release ../paraview
make -j$(nproc)

And finally the dockerfile from the directory with the trame application.

FROM paraview-trame AS trame
RUN install -d -o trame-user -g trame-user /deploy
ENV TRAME_PARAVIEW=/opt/paraview
ENV TRAME_CLIENT_TYPE=vue3
COPY --chown=trame-user:trame-user . /deploy
RUN /opt/trame/entrypoint.sh build

[jourdain]

I guess the reason why you aim to build ParaView is because you are on a mac arm and need a linux-arm build of ParaView.
Otherwise you should just download and unzip our package of ParaView.

You need to build the superbuild version of ParaView or you will be missing some Python pieces.
Also after building, you should have an install step and that should be the piece you copy.

At that point you should ask such question on ParaView discourse. (How to build ParaView within docker with ...)

[smkacur]

Hi Seb,
I had more time to look into this and you are right, my my last post was conflating the paraview build with the problems that I am having when modifying the simple paraview/Docker deployment that I originally posted about.

When I am on my m1 mac and am using this dockerfile I can run the paraview/Docker example with no issue.

FROM --platform=linux/amd64 kitware/trame:py3.9

RUN install -d -o trame-user -g trame-user /deploy
RUN apt update -y && apt install -y libosmesa6-dev

ARG PV_URL='https://www.paraview.org/files/v5.11/ParaView-5.11.0-osmesa-MPI-Linux-Python3.9-x86_64.tar.gz'
RUN mkdir -p /opt/paraview && cd /opt/paraview && wget -qO- $PV_URL | tar --strip-components=1 -xzv
ENV TRAME_PARAVIEW=/opt/paraview
ENV TRAME_CLIENT_TYPE=vue3
COPY --chown=trame-user:trame-user . /deploy

RUN /opt/trame/entrypoint.sh build

However, when I update the setup/requirements.txt to include pandas and then try to import pandas into the example application. I see this error in the container logs:

2024-05-07 08:53:10 Starting server...
2024-05-07 08:53:10 Starting the wslink launcher at
2024-05-07 08:53:11 ======== Running on http://0.0.0.0:9000 ========
2024-05-07 08:53:11 (Press CTRL+C to quit)
2024-05-07 08:53:39 Error handling request
2024-05-07 08:53:39 Traceback (most recent call last):
2024-05-07 08:53:39   File "/deploy/server/venv/lib/python3.9/site-packages/aiohttp/web_protocol.py", line 452, in _handle_request
2024-05-07 08:53:39     resp = await request_handler(request)
2024-05-07 08:53:39   File "/deploy/server/venv/lib/python3.9/site-packages/aiohttp/web_app.py", line 543, in _handle
2024-05-07 08:53:39     resp = await handler(request)
2024-05-07 08:53:39   File "/deploy/server/venv/lib/python3.9/site-packages/wslink/backends/aiohttp/launcher.py", line 97, in handle_post
2024-05-07 08:53:39     return await self._waitForReady(session, request)
2024-05-07 08:53:39   File "/deploy/server/venv/lib/python3.9/site-packages/wslink/backends/aiohttp/launcher.py", line 109, in _waitForReady
2024-05-07 08:53:39     if self.process_manager.isReady(session, count):
2024-05-07 08:53:39   File "/deploy/server/venv/lib/python3.9/site-packages/wslink/launcher.py", line 480, in isReady
2024-05-07 08:53:39     if not self.isRunning(id) and count < 60:
2024-05-07 08:53:39   File "/deploy/server/venv/lib/python3.9/site-packages/wslink/launcher.py", line 468, in isRunning
2024-05-07 08:53:39     return self.processes[id].poll() is None
2024-05-07 08:53:39 KeyError: '4ffa0d78-0c92-11ef-aa73-0242ac110002'

And this traceback under /deploy/server/logs/launcher/4ffa0d78-0c92-11ef-aa73-0242ac110002.txt:

Traceback (most recent call last):
  File "/deploy/server/venv/lib/python3.9/site-packages/pandas/__init__.py", line 26, in <module>
    from pandas.compat import (
  File "/deploy/server/venv/lib/python3.9/site-packages/pandas/compat/__init__.py", line 17, in <module>
    from pandas.compat._constants import (
  File "/deploy/server/venv/lib/python3.9/site-packages/pandas/compat/_constants.py", line 20, in <module>
    ISMUSL = "musl" in (sysconfig.get_config_var("HOST_GNU_TYPE") or "")
  File "/opt/paraview/lib/python3.9/sysconfig.py", line 615, in get_config_var
    return get_config_vars().get(name)
  File "/opt/paraview/lib/python3.9/sysconfig.py", line 564, in get_config_vars
    _init_posix(_CONFIG_VARS)
  File "/opt/paraview/lib/python3.9/sysconfig.py", line 430, in _init_posix
    _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
ModuleNotFoundError: No module named '_sysconfigdata__linux_x86_64-linux-gnu

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/deploy/SimpleCone.py", line 5, in <module>
    import pandas as pd
  File "/deploy/server/venv/lib/python3.9/site-packages/pandas/__init__.py", line 31, in <module>
    raise ImportError(
ImportError: C extension: _sysconfigdata__linux_x86_64-linux-gnu not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext' to build the C extensions first.

Do you have any advice on troubleshooting or a workaround? I use pandas to manipulate some environmental data (precipitation, temperature, etc.) for plotting with plotly alongside the paraview visual.

[jourdain]

ParaView bundle pandas and here you have a conflicting version issue. You could remove the pandas directory from ParaView to maybe grab a newer version, or just assume it will be there thanks to ParaView.

[smkacur]

Setting the pandas version to the same one installed by paraview solved this. Thank you for your help!