Skip to content

Conversation

@prckent
Copy link
Contributor

@prckent prckent commented Jan 13, 2026

Proposed changes

Update the tests/estimator/* tests.

Multiple local runs show these tests to be deterministic. Wider range of build configurations may find we need to revise error bars etc.

What type(s) of changes does this code introduce?

  • Testing changes (e.g. new unit/integration/performance tests)

Does this introduce a breaking change?

  • No (hopefully)

What systems has this change been tested on?

mac, container, cpu full precision only

Checklist

    • I have read the pull request guidance and develop docs
    • This PR is up to date with the current state of 'develop'
    • Code added or changed in the PR has been clang-formatted
    • This PR adds tests to cover any new code, or to catch a bug that is being fixed
    • Documentation has been added (if appropriate)

@prckent
Copy link
Contributor Author

prckent commented Jan 13, 2026

Removed the acforce estimators since these were wildly different. Potentially special handling needed for mixed precision?
e.g. (GCC12-NoMPI-Werror-Real-Mixed)

785/1009 Test   #94: deterministic-estimator-acforce-legacy-1-16-check ....................................................***Failed    0.15 sec
Error.  LocalEnergy Ref =  -15.586712248  Val =  -15.309821907
Error.  Kinetic Ref =  12.734901025  Val =  11.739694886
Error.  NonLocalECP Ref =  1.8493765281  Val =  1.3342837611
Error. ACForce_0_0 Ref =  0.49376340185  Val =  0.05556758680000007
Error. ACForce_Ewfgrad_0_0 Ref =  -4.2101117647  Val =  -3.760228015
Error. ACForce_wfgrad_0_0 Ref =  0.30004600095  Val =  0.25512302294
Error. ACForce_0_1 Ref =  0.35860558777  Val =  -0.04760729522000001
Error. ACForce_Ewfgrad_0_1 Ref =  2.3849276626  Val =  4.2054679351
Error. ACForce_wfgrad_0_1 Ref =  -0.18114010291  Val =  -0.2904528128
Error. ACForce_0_2 Ref =  5.2288476311  Val =  4.544800507801001
Error. ACForce_Ewfgrad_0_2 Ref =  24.54222043  Val =  21.983831041
Error. ACForce_wfgrad_0_2 Ref =  -1.5614748618  Val =  -1.4511443451
Error. ACForce_1_0 Ref =  1.1912085227  Val =  -0.7618839144
Error. ACForce_Ewfgrad_1_0 Ref =  -8.1699621441  Val =  4.3089152784
Error. ACForce_wfgrad_1_0 Ref =  0.33676305176  Val =  -0.28127478622
Error. ACForce_1_1 Ref =  4.764544717  Val =  0.4219537899000001
Error. ACForce_Ewfgrad_1_1 Ref =  -17.347544391  Val =  0.031055022025
Error. ACForce_wfgrad_1_1 Ref =  0.83752812817  Val =  -0.0066919000819
Error. ACForce_1_2 Ref =  -0.041953731361  Val =  -4.04491188564
Error. ACForce_Ewfgrad_1_2 Ref =  -46.64908178  Val =  -26.006980804
Error. ACForce_wfgrad_1_2 Ref =  2.6949237554  Val =  1.6685325466
...
 795/1009 Test   #96: deterministic-estimator-acforce-fast-1-16-check ......................................................***Failed    0.15 sec
Error.  LocalEnergy Ref =  -15.586712248  Val =  -15.309822125
Error.  Kinetic Ref =  12.734901025  Val =  11.73966416
Error.  NonLocalECP Ref =  1.8493765281  Val =  1.3342956033
Error. ACForce_0_0 Ref =  0.49376340185  Val =  0.055575130507
Error. ACForce_Ewfgrad_0_0 Ref =  -4.2101117647  Val =  -3.7603102827
Error. ACForce_wfgrad_0_0 Ref =  0.30004600095  Val =  0.25512862206
Error. ACForce_0_1 Ref =  0.35860558777  Val =  -0.047622982413
Error. ACForce_Ewfgrad_0_1 Ref =  2.3849276626  Val =  4.2056168073
Error. ACForce_wfgrad_0_1 Ref =  -0.18114010291  Val =  -0.29046243429
Error. ACForce_0_2 Ref =  5.2288476311  Val =  4.5448083878
Error. ACForce_Ewfgrad_0_2 Ref =  24.54222043  Val =  21.98370087
Error. ACForce_wfgrad_0_2 Ref =  -1.5614748618  Val =  -1.4511358514
Error. ACForce_1_0 Ref =  1.1912085227  Val =  -0.76189327613
Error. ACForce_Ewfgrad_1_0 Ref =  -8.1699621441  Val =  4.30898566
Error. ACForce_wfgrad_1_0 Ref =  0.33676305176  Val =  -0.28127895016
Error. ACForce_1_1 Ref =  4.764544717  Val =  0.4219654128
Error. ACForce_Ewfgrad_1_1 Ref =  -17.347544391  Val =  0.031118970483
Error. ACForce_wfgrad_1_1 Ref =  0.83752812817  Val =  -0.0066962894052
Error. ACForce_1_2 Ref =  -0.041953731361  Val =  -4.0448967814
Error. ACForce_Ewfgrad_1_2 Ref =  -46.64908178  Val =  -26.006828673
Error. ACForce_wfgrad_1_2 Ref =  2.6949237554  Val =  1.6685216241

Copy link
Contributor

@ye-luo ye-luo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given CI ends up happy

@ye-luo
Copy link
Contributor

ye-luo commented Jan 13, 2026

Test this please

@ye-luo ye-luo enabled auto-merge January 13, 2026 23:49
@prckent
Copy link
Contributor Author

prckent commented Jan 14, 2026

  1. Interesting the CI is failing due to absence of pandas on the relevant machines. The check_python_reqs() is supposed to protect against this -> there are bugs that Remove pandas dependency from QMCPACK tests #5732 removes the requirement to even have. Unfortunately RHEL9 doesn't obviously ship python3-pandas => merge 5732 and revisit this.

  2. Gcov shows appropriately increased coverage.

@prckent
Copy link
Contributor Author

prckent commented Jan 15, 2026

The few failures previously observed are now gone (?)

@prckent
Copy link
Contributor Author

prckent commented Jan 15, 2026

@ye-luo : I have not been able to reproduce the previously seen failures either when running the container used for CI or on my own systems (x86,arm). However, checking cdash shows some occasional failures ( https://cdash.qmcpack.org/testSummary.php?project=1&name=estimator-latdev-free-1-16-check&date=2026-01-15 ) in the nightlies. Average ~1/day. The test error output is meaningless out of context, so I will investigate. Probably need to remake these tests completely, making them shorter in particular. This update to include the tests in the CI will therefore wait.

@prckent prckent marked this pull request as draft January 15, 2026 13:53
auto-merge was automatically disabled January 15, 2026 13:53

Pull request was converted to draft

@prckent prckent changed the title Rename and relabel estimator tests as deterministic [WIP] Rename and relabel estimator tests as deterministic Jan 15, 2026
@ye-luo
Copy link
Contributor

ye-luo commented Jan 15, 2026

@prckent let us move the unstable ones out of determinsitic tests and merge this PR. In the meantime, open an issue to track this tests as we troubleshoot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants