Run Pre-built Binary

[michaelmckinsey1]

Description

resolves #609
This PR enables running an experiment using an existing binary, i.e. not requiring building with spack. The environment-modules package manager is used instead, for ramble to load necessary modules for the binary. Includes a refactor of the compute_spack_section() function, into a generalized compute_package_section(), such that other package managers can share the same functionalities.

• Test benchmark binaries:
  • osu-micro-benchmarks
  • stream
• Do not clone spack if the package manager is not spack
• Add CI
• Add docs

Adding/modifying a benchmark (docs: Adding a Benchmark)

• Modify lib/benchpark/experiment.py to enable environment-modules package manager and appending the binary location to PATH.
• Modify lib/benchpark/cmd/setup.py to not setup spack if package manager is not spack.
• Modify experiments/*.py to refactor compute_package_section() function.

Example usage

Expected binary osu_latency in provided external path /usr/workspace/wsa/mckinsey/osu-micro-benchmarks/mpi/pt2pt:

benchpark experiment init --dest=osumb osu-micro-benchmarks package_manager="environment-modules" workload="osu_latency" append_path="/usr/workspace/wsa/mckinsey/bin/osu-micro-benchmarks/mpi/pt2pt"
benchpark setup ./osumb/ ./ruby-system/ osumb-ruby/

results in new clause in experiment ramble.yaml:

env_vars:
    append:
    - paths:
        PATH: /usr/workspace/wsa/mckinsey/osu-micro-benchmarks/mpi/pt2pt
    set: {}

execute_experiment will now include:

export PATH="${PATH}:/usr/workspace/wsa/mckinsey/bin/osu-micro-benchmarks/mpi/pt2pt";
. /usr/WS1/mckinsey/benchpark/osumb-ruby/osumb/LlnlCluster-9751dbe/workspace/software/environment-modules/osu-micro-benchmarks/module_loads
srun -N 2 osu_latency ...

[michaelmckinsey1]

Ignore this comment. Current version requires no changes to ramble.

This currently requires a change to ramble that I don't know how to avoid (to prepend the path in execute_experiment).

+++ b/var/ramble/repos/builtin/package_managers/environment-modules/package_manager.py
@@ -95,6 +95,14 @@ class EnvironmentModules(PackageManagerBase):
         with open(module_file_path, "w+") as f:
             f.write(loads_content)
 
+    register_builtin("add_to_path")
+
+    def add_to_path(self):
+        paths = []
+        for p in self.app_inst.variables["binary_path"]:
+            paths.append("PATH="+p+":"+"$PATH")
+        return paths
+
     register_builtin("module_load", required=True)
 
     def module_load(self):

[pearce8]

@scheibelp @becker33 Please assess the design. Also please suggest how we can prepend the path in execute_experiment - or help formulate a question for Doug.

[michaelmckinsey1]

@scheibelp @becker33 Please assess the design. Also please suggest how we can prepend the path in execute_experiment - or help formulate a question for Doug.

The path part has been figured out. We are using an existing ramble interface to append to the path.

[scheibelp]

It looks like all of these individual subclasses of Experiment could be left alone, and you could skip compute_spack_section in compute_package_section_wrapper if package_manager is not spack. Am I missing something? It looks like all these functions are turned effectively into noops when the package manager is not spack, so it seems like it would make sense to skip the function altogether.

[michaelmckinsey1]

Good point. Upon checking, the environment-modules manager in ramble actually does need this information, to create the file where modules are loaded. I removed the if-statements in the subclasses 313ff73. Having the package_specs be empty for non-spack managers was a mistake.

[pearce8]

1. Please add a test of a binary in dry runs. Maybe build/run saxpy as is now, then run that binary?
2. Please add a docs page showing your example. Maybe call the page "Running a binary using Benchpark" and put it last under the "Running Benchpark" section.

[michaelmckinsey1]

1. Please add a test of a binary in dry runs. Maybe build/run saxpy as is now, then run that binary?

   2. Please add a docs page showing your example. Maybe call the page "Running a binary using Benchpark" and put it last under the "Running Benchpark" section.

1. Done in efa22ab
2. 13e7d0c