Missing requirements.txt and venv in some Python-based examples

[MakisH]

A generalization of #190.

Everything Nutils-based is already covered. We need to adapt our FEniCS and all Python-only examples.

[BenjaminRodenberg]

A list of open ToDos:

• https://github.com/precice/tutorials/tree/develop/channel-transport-reaction/chemical-fenics
• https://github.com/precice/tutorials/tree/develop/channel-transport-reaction/fluid-fenics
• https://github.com/precice/tutorials/tree/develop/elastic-tube-1d/fluid-python Add requirements.txt to elastic-tube-1d #564 @MakisH
• https://github.com/precice/tutorials/tree/develop/elastic-tube-1d/solid-python Add requirements.txt to elastic-tube-1d #564 @MakisH
• https://github.com/precice/tutorials/tree/develop/elastic-tube-3d/solid-fenics
• https://github.com/precice/tutorials/tree/develop/flow-around-controlled-moving-cylinder/solid-python
• https://github.com/precice/tutorials/tree/develop/flow-around-controlled-moving-cylinder/controller-fmi fmi @uekerman
• https://github.com/precice/tutorials/tree/develop/flow-over-heated-plate/solid-fenics
• https://github.com/precice/tutorials/tree/develop/oscillator/mass-left-fmi fmi @uekerman
• https://github.com/precice/tutorials/tree/develop/oscillator/mass-right-fmi fmi @uekerman
• Add missing requirements.txt to oscillator and oscillator-overlap tutorial #556
  • https://github.com/precice/tutorials/tree/develop/oscillator/mass-right-python
  • https://github.com/precice/tutorials/tree/develop/oscillator/mass-left-python
  • https://github.com/precice/tutorials/tree/develop/oscillator-overlap/mass-left-python
  • https://github.com/precice/tutorials/tree/develop/oscillator-overlap/mass-right-python
• https://github.com/precice/tutorials/tree/develop/partitioned-heat-conduction-complex/dirichlet-fenics
• https://github.com/precice/tutorials/tree/develop/partitioned-heat-conduction-complex/neumann-fenics
• https://github.com/precice/tutorials/tree/develop/partitioned-heat-conduction-overlap/left-fenics
• https://github.com/precice/tutorials/tree/develop/partitioned-heat-conduction-overlap/right-fenics
• 3a00c01 in pySDC & FEniCS solver for partitioned-heat-equation #557
  • https://github.com/precice/tutorials/tree/develop/partitioned-heat-conduction/dirichlet-fenics
  • https://github.com/precice/tutorials/tree/develop/partitioned-heat-conduction/neumann-fenics
• https://github.com/precice/tutorials/tree/develop/perpendicular-flap/solid-fenics
• Add python variant of resonant circuit #498
• https://github.com/precice/tutorials/tree/develop/volume-coupled-diffusion/drain-fenics
• https://github.com/precice/tutorials/tree/develop/volume-coupled-diffusion/source-fenics

[BenjaminRodenberg]

That's actually quite a long list of cases that need an update. For the pure python / FEniCS cases the procedure is clear. Regarding the FMI cases some input from @uekerman would be appreciated since I'm not that much into the topic.

[BenjaminRodenberg]

@MakisH suggested to not put the requirements.txt and venv inside the participant directory but rather into the solver directory. I.e. not structure for the oscillator as given in #556:

oscillator
├── mass-left-python
│   ├── requirements.txt  *
│   ├── run.sh
│   └── .venv  *
├── mass-right-python
│   ├── requirements.txt  *
│   ├── run.sh
│   └── .venv  *
└── solver-python
    └── oscillator.py

Instead we can use:

oscillator
├── mass-left-python
│   └── run.sh
├── mass-right-python
│   └── run.sh
└── solver-python
    ├── requirements.txt
    ├── .venv
    └── oscillator.py

The files/folders marked with a * are duplicates and, therefore, the second solution would save us a lot of implementation and maintainment work.

Note: I think this would also apply to the nutils cases and would justify introducing a solver-nutils directory containing the requirements.txt file.

[MakisH]

Note: I think this would also apply to the nutils cases and would justify introducing a solver-nutils directory containing the requirements.txt file.

Possibly, yes, but let's keep these as independent PRs. Let's focus on getting requirements.txt everywhere first. Whenever we open the box of "what is similar enough to be merged", that will be a long path.

[BenjaminRodenberg]

Note: I think this would also apply to the nutils cases and would justify introducing a solver-nutils directory containing the requirements.txt file.

Possibly, yes, but let's keep these as independent PRs. Let's focus on getting requirements.txt everywhere first. Whenever we open the box of "what is similar enough to be merged", that will be a long path.

@MakisH True. #498 is already an exception: The requirements.txt files for the two python participants should be identical, but there is no solver folder and the participants actually have independent scripts. It's just coincidence they have identical dependencies. Let's focus on the first step as you suggested.

Edit: Or are you only referring to the nutils cases with this comment?

[MakisH]

If there is a common folder (e.g., solver-python), then I would add the venv there. If not, I would not force one just for the dependencies of each script, even if two scripts happen to have the same dependencies at the moment. That would be unnecessary coupling.

But with the "let's not open that box", I was referring to Nutils.

[BenjaminRodenberg]

If there is a common folder (e.g., solver-python), then I would add the venv there. If not, I would not force one just for the dependencies of each script, even if two scripts happen to have the same dependencies at the moment. That would be unnecessary coupling.

But with the "let's not open that box", I was referring to Nutils.

Sounds good. 👍 Then I think we can start implementing 🎉 Already discussed this with @NiklasVin. He will take care of all the necessary updates.

[uekerman]

Regarding the FMI cases some input from @uekerman would be appreciated since I'm not that much into the topic.

Only requiring fmiprecice should do the job, no? Everything else is a dependency of this package.

[BenjaminRodenberg]

For the FEniCS cases I just ran into the the problem that accessing FEniCS from a venv does not work (https://fenicsproject.discourse.group/t/using-fenics-with-python-venv/4887).

Using anaconda and the corresponding FEniCS package (see https://fenicsproject.org/download/archive/) should be possible. We also have the fenics adapter and the python bindings on anaconda (see https://anaconda.org/conda-forge/fenicsprecice). But this is again a completely different approach...

[BenjaminRodenberg]

Seems like python3 -m venv --system-site-packages ../solver-fenics/.venv does the trick (assuming FEniCS was installed using apt). I think this is a proper solution. See 3a00c01.

[BenjaminRodenberg]

I think python3 -m venv ../solver-fenics/.venv might be a bad idea after all. See #567.

[fsimonis]

The easy way out is to keep all solver information in the solver directory including requirements.txt, and use a local venv in the participant directory where the run script is.

pip uses a user-local cache for requests even when using a venv, so this shouldn't be too much of an issue. pip cache dir

The problem with the current layout is that run scripts are pretty ugly.
A pretty straight-forward solution for this is to make solvers into mini packages.

Example for the oscillator:

This needs a subdirectory as it contains multiple files. Single-file packages don't need this extra step.

oscillator/solver-python
├── oscillator
│   ├── oscillator.py
│   ├── problemDefinition.py
│   └── timeSteppers.py
└── pyproject.toml

[project]
name = "oscillator"
version = "0" # we don't care
dependencies = [
    "numpy",
    "pyprecice~=3.0",
    "scipy"
]

# Defining entry points that are available in PATH after installation.
# This is practical for calling them and killing them by name.
[project.scripts]
leftMass = "oscillator.oscillator:runLeft"
rightMass = "oscillator.oscillator:runRight"

from . import problemDefinition
from . import timeSteppers

class Participant(Enum):
    MASS_LEFT = "Mass-Left"
    MASS_RIGHT = "Mass-Right"


def runLeft():
    run(Participant.MASS_LEFT.value)


def runRight():
    run(Participant.MASS_RIGHT.value)


def run(participant_name):
    parser = argparse.ArgumentParser()
    # ...

[BenjaminRodenberg]

Thanks for the input @fsimonis! Modularizing some of the python-based solvers better would definitely help. I would still suggest to first go for the simple approach and live with a global requirements.txt in the solver directory and create the venv in the participant directory. Creating proper packages for the solvers I see as a second step that we could add later (created an issue #568).