Paraview recipe (#39)

* [email protected] build on hohgant with new EGL support

* osmesa debugging

* Enable build-cache mirror

* Add job script for paraview build

* More tweaks to try to fix osmesa on hohgant

* Moving/renaming files to fit alps recipes repo structure

* Update convenience build scripts for debugging

* Use spack release version, with paraview customizations in stackinator repo section

* use architecture dirs (eg a100) for different recipes

* Recipe for minimal paraview build on clariden

* Use symlinks for repos to avoid duplication, use copy.yaml for patch replication

copy.yaml inside a packae repo tells stackinator to copy patch files
from the spack source into the user repo to save the user needing to
mange it by hand.

* Add stackinator build and paraview client config and pvserver launch scripts

These scripts will need to be further customized for use with
stackinator based installations of paraview.

* Paraview build recipe updated with EGL, experimental plugin support

Lots of utility scripts and other ancillary files also commmitted in
scripts/ dir

* Sync paraview package with current EGL branch in spack

* Bump paraview to latest 5.11.2 version

* Cleanup scripts

* Paraview EGL now part of spack master, delete local repos, update recipes

* Add santis recipe and build script

* New hybrid approach to paraview install using cmake directly

Install dependencies using spack, but check out custom branches
of paraview / VTK / vtkm to work around compilation issues with the
default spack available branches.
Compile using cmake and a build-env generated by creating a fake package
with all the dependencies we need added into it and concretized with the
original spack/stackinator generated environment

* libtree, clean build and permissions updates

* Remove variant gl=egl in favour of virtual gl=egl

* replace santis with todi

* Disable modules by default for gh200 paraview builds

* Fix version of stdexec to pika compatible sha

* [email protected]

* Lots of random fixes for paraview + scripts

* Fix eiger paraview stack using osmesa and lib cdi support

* Fix link error for ispc

* bump paraview to 5.13, disable raytracing on gh200 for now

* Cleanup scripts/paths after rebase, add pvserver setup file etc

* Fix TBB find problem by setting TBB_ROOT

* Add cdi from albestro/alby/paraview instesad of c2sm to avoid version incompatibility

* Use jfrog for license. Fix cdi, adios, plugin builds and submit scripts

* Fix paraview python view setup for numpy etc

* rename amd-zen2 to plain zen2, fix scripts paths, names, etc

* Add cluster name to RC launch script and cleanup script comments etc

* delete old/unused c2sm libs

* Rename todi to daint

* change directory structure

* update config.yaml

* Fix build script paths after version added to recipe location

* Set spack commit to recent SHA and get paraview source from jfrog

* update paraview to 5.13.1, add adios2, h5py and conduit

* Clean up build-env for plugins, add h5py for pv-python

* Remove h5hut from gh200 recipe

* Remove ispc customizations and set spack develop tag

* fix for spack@develop

* paraview: cleanup views in environment setup

* paraview: update 5.13.1 -> 5.13.2

* Improve user auto-config scripts for rc-server setup etc

* Update zen2 recipe with recent changes to install scripts

* Cleanup commits to ensure we always use a fixed version/commit

* Fix import from spack packages in env-to-package script

* Fix libtheora hash as using @master introduced new build error

* Update recipes/paraview/5.13.2/gh200/config.yaml

Co-authored-by: Mikael Simberg <[email protected]>

* Remove cache-config from recipe repo

* Add clariden/santis to paraview gh200 build list

* Remove nvindex config yaml file as it is in jfrog now

* Remove old sync/update script

* Rename environments for simplicity

* Add note about git spack SHA used for recipe

* Remove some packages used in debugging and building other plugins etc

* More cleanup of build/src folders after install

* remove modules.yaml as it is not used

* bump boootstrap to gcc@12

* Fix gcc and env setup for post install scripts broken by last few commits

* Add set-e to post install scripts to make sure that if build fails, uenvs are not stored

* Fixes to post install to ensure build completes ok

* Change nvindex plugin tarball to aarch64 version for gh200

* Fix jfrog nvidia index access for aarch64 and regular linux files

* fix generated nvindex lib path - remove unused reframe yaml

---------

Co-authored-by: Simon Pintarelli <[email protected]>
Co-authored-by: Mikael Simberg <[email protected]>

Author: 3 people

config.yaml

@@ -413,3 +413,14 @@ uenvs:
         clariden: [gh200]
         eiger: [zen2]
       mount: "/user-tools"
+  paraview:
+    "5.13.2":
+      recipes:
+        gh200: 5.13.2/gh200
+        zen2: 5.13.2/zen2
+      deploy:
+        daint: [gh200]
+        clariden: [gh200]
+        santis: [gh200]
+        eiger: [zen2]
+      develop: True

recipes/paraview/5.13.2/gh200/compilers.yaml

@@ -0,0 +1,5 @@
+bootstrap:
+  spec: [email protected]
+gcc:
+  specs:
+  - [email protected]

recipes/paraview/5.13.2/gh200/config.yaml

@@ -0,0 +1,8 @@
+# NB. git sha for spack is required for some vtk external packages, but newer sha breaks other stuff
+name: ParaView
+store: /user-environment
+spack:
+  commit: 195b869e1c5f0f850fe16ca1cc0c77fc71f5fcd6
+  repo: https://github.com/spack/spack.git
+description: ParaView for Grace-Hopper using EGL
+modules: false

recipes/paraview/5.13.2/gh200/environments.yaml

@@ -0,0 +1,127 @@
+paraview-cuda:
+  compiler:
+  - toolchain: gcc
+    spec: gcc
+  mpi:
+    spec: cray-mpich
+    gpu: cuda
+  unify: true
+  views:
+    develop:
+      exclude: []
+      uenv:
+        add_compilers: true
+        prefix_paths:
+          LD_LIBRARY_PATH: [lib, lib64]
+    paraview:
+      link: run
+      exclude: []
+      uenv:
+        add_compilers: true
+        prefix_paths:
+          LD_LIBRARY_PATH: [lib, lib64]
+    paraview-python:
+      link: run
+      exclude: []
+      uenv:
+        prefix_paths:
+          LD_LIBRARY_PATH: [lib, lib64]
+
+  specs:
+
+  # cuda
+  - [email protected]
+
+  # Build tools
+  # - git-lfs
+  - cmake
+  - ninja
+  - direnv
+  - ninja
+  - libtree
+  - llvm@18 ~gold ~cuda
+
+  # maths
+  - blaspp
+  - eigen
+  - fftw
+  - lapackpp
+  - openblas
+  - proj
+
+  # c++ helpers
+  - fmt@10
+  - boost +atomic +chrono +container +context +coroutine +date_time +filesystem +graph +json +mpi +multithreaded +program_options +regex +serialization +shared +system +test +thread
+
+  # IO and parallelism
+  - hdf5 +mpi +cxx +hl +threadsafe +shared ~java
+  - netcdf-c +mpi
+  - lz4
+  - adios2 +python +hdf5
+
+  # memory management
+  # - jemalloc
+  # - mimalloc
+
+  # # system
+  # - numactl
+
+  # # profiling/testing
+  # - gperftools
+  # - googletest
+
+  # multithreading
+  - tbb
+
+  # in-situ support
+  - libcatalyst +mpi +python
+
+  # vtk external deps
+  - [email protected]
+  - [email protected]
+  - gl2ps
+  - glew
+  - jpeg
+  - jsoncpp
+  - libharu
+  - libtiff
+  - nlohmann-json
+  - [email protected]=master
+  - pugixml
+  - pegtl
+  - protobuf@:3.21
+  - seacas ~fortran ~applications ~legacy ~tests ~x11
+  - utf8cpp
+
+  # climate/weather
+  - cdi
+
+  # python
+  - [email protected]
+  - py-numpy
+  - py-pandas
+  - py-matplotlib
+  - py-mpi4py
+  - py-cftime
+  - py-h5py
+
+  # raytracing in VTK/ParaView
+  - [email protected] ~mpi +denoiser +volumes ~apps ~glm
+  - [email protected]
+  - openvkl
+  - embree
+  - rkcommon
+  - openimagedenoise
+
+  variants:
+  - +cuda
+  - cuda_arch=90
+  - ^[virtuals=gl] egl
+  - build_type=Release
+  - cxxstd=17
+  - +mpi
+  - ~fortran
+  - ~x11
+  - ~examples
+  - ~tests
+  - +cxx

recipes/paraview/5.13.2/gh200/packages.yaml

@@ -0,0 +1,11 @@
+packages:
+  all:
+    providers:
+      gl: [egl]
+  egl:
+    externals:
+    - spec: [email protected]
+      prefix: /usr
+    buildable: false
+  llvm:
+    require: llvm ~gold ~cuda