Create script for automated regression testing #13
Conversation
There was a problem hiding this comment.
In terms of functionality, this works great! A few suggestions in the review.
I also noticed that the results change based on compilation options, e.g. when compiling for sp2n (--enable-Sp --enable-Nc=4) and running Test_hmc_IwasakiGauge I got different results than the same test when compiled for su3 (--enable-Nc=3). I'm not sure these values will make a lot of sense without a very strict compile and run parameters.
| @@ -0,0 +1 @@ | |||
| 8.8.8.8 1.1.1.1 0.0256253844 922c392f d1e4cc1c | |||
There was a problem hiding this comment.
| 8.8.8.8 1.1.1.1 0.0256253844 922c392f d1e4cc1c | |
| 8.8.8.8 1.1.1.1 0.281507042 922c392f 716ec776 |
There was a problem hiding this comment.
Updated expected output to align with original test (MDsteps = 36 and trajL = 1.0).
| TheHMC.Parameters.MD.MDsteps = 2; | ||
| TheHMC.Parameters.MD.trajL = 0.1; |
There was a problem hiding this comment.
| TheHMC.Parameters.MD.MDsteps = 2; | |
| TheHMC.Parameters.MD.trajL = 0.1; | |
| TheHMC.Parameters.MD.MDsteps = 36; | |
| TheHMC.Parameters.MD.trajL = 1.0; |
There was a problem hiding this comment.
I think it would be best to to change the test case in the develop branch as it should stay consistent with the upstream.
There was a problem hiding this comment.
I hope you mean "not to change the test case...." :) . The problem is that the expected value is for the modified MDsteps and trajL. Do we want to ditch them completely? (I saw you fixed those in the expected values file below)
|
Something I should have considered earlier: since this uses a Metropolis algorithm, checking the final value of the plaquette is only valid if the version being compared against accepts the change. (In the case of a rejected change, then the value of the plaquette output is the input value, and only depends on the code we are touching if it changes in such a way that the change is accepted instead, which is unlikely.) Provided we consistently stick with the same test case (and have verified that it accepts), then I think this is fine, but we should be careful about generalising this. |
Co-authored-by: Mashy Green <[email protected]>
|
Fixes #8 .
This is stil WIP, I need to
builddirectory and paths are correctpytestBut you can already use it from inside the directory that the test sits in
build, after copying the file with the reference values over.@edbennett are the values we're checking the right ones? Or do we need the palette value to a gazillion decimals?