regression test readme

Author: calccrypto

test/regression/README.md

@@ -0,0 +1,59 @@
+# Regression Tests
+
+This directory contains regression tests for GUFI.
+
+## Usage
+
+Each regression test consists of at least two files: a bash script and
+a corresponding .expected file. There might be additional files, such
+as Python scripts, associated with each test. To run a single set of
+tests, run the script file from a directory that allows for extended
+attributes, and it should work. The output will be printed to standard
+out as well as a corresponding .out file. The diff between the
+.expected file and .outfile will also be printed. The regression tests
+can be viewed as known answer tests.
+
+If a regression test's .expected needs to be updated with the latest
+output from the test script, the .out file can be usd to directly
+overwrite the .expected file with `cp`, `mv`, or `cat` and a redirect.
+
+## Common Files
+
+- `setup.sh` contains common functions, variables, cleanup code, etc.
+    - `replace`: replaces environment-specific text with common text,
+      such as the actual path to an executable with the name
+
+    - `run_sort`: runs the given command, printing the command as well
+      as the sorted output, passed into `replace`
+
+    - `run_no_sort`: runs the given command, printing the command as
+      well as the unsorted output, passed into `replace`
+
+    - `setup_cleanup`: function that is called on exit of script
+        - wrap this with a test-specific cleanup function if the test
+          requires custom cleanup
+
+- `generatetree.sh` generates a fixed tree with known properties
+    - This is the most commonly used script/tree
+
+- `rollup_tree.sh` generates a fixed tree with known properties for
+  testing rollups with
+    - Requires `sudo`
+
+- `gufi_tool.py` fakes entry points into Python tools in order to use
+  a test configuration file instead of the real one
+
+## Test Script Structure
+1. `set -e`
+2. `. setup.sh 1`
+    - Pass in 0 if the source tree and index do not need to be generated
+3. Define variables
+4. If necessary, set up cleanup function and call it
+5. Define the name where the output will be written to
+    - `OUTPUT="<name>.out"`
+6. Define functions
+7. Run tests in a subshell
+    - `tee` all output to `"${OUTPUT}"`
+8. Diff the expected file with `"${OUTPUT}"`
+9. Remove `"${OUTPUT}"`
+    - This only happens if the diff in the previous step succeeded