add debug helper script and docs (eth-cscs#114)

Author: bcumming

Diff for: docs/development.doc

@@ -0,0 +1,19 @@
+# Development
+
+This page is for developers and maintainers of Stackinator.
+
+## Debug environment
+
+Debugging stack builds can be challenging, because the build uses an environment with paths mounted and remounted using bwrap and different environment variables than the calling shell.
+
+A helper script that will open a new shell with the same environment as the stack build is generated in the build path.
+The script, `stack-debug.sh`, can be sourced to start the new bash shell:
+
+```bash
+user@hostname:/dev/shm/project-build > source ./stack-debug.sh
+build-env >>>
+```
+
+The new shell has `spack` in its path, and has the store path mounted at the environment's mount point.
+To finish debugging, exit the shell with `exit` or ctrl-d.
+

Diff for: mkdocs.yml

@@ -9,6 +9,7 @@ nav:
     - 'Cluster Configuration': cluster-config.md
     - 'Interfaces': interfaces.md
     - 'Build Caches': build-caches.md
+  - 'Development': development.md
 # - Tutorial: tutorial.md
 theme:
   name: material

Diff for: stackinator/builder.py

@@ -360,3 +360,14 @@ def generate(self, recipe):
             self._logger.debug(f"copying extra recipe meta data to {meta_extra_path}")
             shutil.copytree(recipe.user_extra, meta_extra_path)
 
+        # create debug helper script
+        debug_script_path = self.path / "stack-debug.sh"
+        debug_script_template = env.get_template("stack-debug.sh")
+        with debug_script_path.open("w") as f:
+            f.write(
+                debug_script_template.render(
+                    mount_path=recipe.config["store"], build_path=str(self.path), verbose=False
+                )
+            )
+            f.write("\n")
+

Diff for: stackinator/etc/bwrap-mutable-root.sh

@@ -7,4 +7,5 @@ for d in /*; do
         args+=("--dev-bind" "$d" "$d")
     fi
 done
-bwrap "${args[@]}" "$@"
+PS1="\[\e[36;1m\]build-env >>> \[\e[0m\]" bwrap "${args[@]}" "$@"
+

Diff for: stackinator/templates/stack-debug.sh

@@ -0,0 +1 @@
+env --ignore-environment PATH=/usr/bin:/bin:{{ build_path }}/spack/bin SOFTWARE_STACK_PROJECT={{ build_path }} STORE={{ mount_path }} SPACK_SYSTEM_CONFIG_PATH={{ build_path }}/config SPACK_USER_CACHE_PATH={{ build_path }}/cache SPACK=spack SPACK_COLOR=always SPACK_USER_CONFIG_PATH=/dev/null LC_ALL=en_US.UTF-8 TZ=UTC SOURCE_DATE_EPOCH=315576060 {{ build_path }}/bwrap-mutable-root.sh --tmpfs ~ --bind {{ build_path }}/tmp /tmp --bind {{ build_path }}/store {{ mount_path }} bash -noprofile -l