[WIP] Degenerated CI Optimization

[NastaMauger]

Proposed changes

This PR is related to an issue where I requested the ability to add constrained optimization for degenerate CI coefficients.

What type(s) of changes does this code introduce?

• New feature
  convert4qmc now accepts the -degenerated argument. A corresponding flag to enable reading this option has also been added.
• Documentation and build changes:
  Updated the convert4qmc usage documentation to include the new -degenerated argument for users.

Does this introduce a breaking change?

• No

Checklist

I would like to take this further by implementing constrained optimization for degenerate CI coefficients. However, I’m currently unsure where in the code this should be done and how to proceed. Any guidance or insights would be greatly appreciated.

Thanks

[ye-luo]

I don't feel the necessity to introduce an option for both converter and qmcpack
If I understand correctly, in #5581, the input style of "Unmodified xml" indicates degenerated=false and your "Modified xml:" indicates degenerated=true.
Then the converter should output using the correct style based on the original gamess run and QMCPACK follows.

Maybe I misunderstood the intention of this added input tag. Let us know.

[NastaMauger]

Hello @ye-luo

I'm not sure I fully understood your comment.

The idea behind this option is to allow (or not allow) constrained optimization. In my case—for benzene—I want to preserve the degeneracy of the CI coefficients (so -degenerated = yes in the xml). However, someone else might want to lift this degeneracy for a specific reason or purpose (-degenerated = no which is the default).

This flexibility not only reduces the amount of modification needed in the code, but also gives users the opportunity to study how maintaining degeneracy in the CI during optimization might impact the results. Because so far, the code has optimized each CI coefficient independently of its degeneracy.

[ye-luo]

How does the multideterminant xml node look like with your change when degenerated=yes?

[NastaMauger]

This current work does not modify the optimization (yet), so the nodes are not affected regardless of whether it is set to true or false.
So far, it only prints the degenerate CSFs.

[jtkrogel]

@NastaMauger Thanks for starting this PR. It looks like a really useful feature to have. Being able to handle/preserve symmetries (#5581 ) is a big step forward.

[NastaMauger]

@jtkrogel I realize now that my title was misleading. The goal is not to preserve the symmetry per se, but rather to preserve the degeneracy of the CSFs. If one wants to use GAMESS alongside GUGA algorithm for specific reasons, it would be beneficial to retain the degeneracy of the corresponding CSFs as found by GAMESS.

So far, QMCPACK treats each CSF as independent, regardless of their coefficients (and thus their degeneracy)

@ye-luo gave some insight into how one might modify the input XML to handle such a case. I'm still concerned, however, about whether this is truly equivalent to a constrained optimization.

[prckent]

Thanks for starting on this. Q. Will searching on equal coefficients suffice? e.g. Is the situation where there are coefficients c and -c infrequent? Some more complex description or grouping of the coefficients could be needed. What is state of the art in different codes?

[NastaMauger]

Hello,
I have finished the code for producing the degenerated XML file, and it works as expected on various cases and systems:

• CSF case (benzene):
  cas_6_6_degenerated.wfj.xml.txt
  cas_6_6_unmodified.wfj.xml.txt
  The Cusp corrections gives an energy of -230.847820 +/- 0.014805 with the degenerated file (reference energy is -230.8620665156 from GAMESS)

• DET case (paraquinone):
  cas_4_4_degenerated.wfj.xml.txt
  cas_4_4_unmodified.wfj.xml.txt
  In this case, the cusp correction using the degenerated file gives -379.400429 +/- 0.019313 (the GAMESS reference is -379.4104497884)

Please let me know if anything else is needed

To answer @prckent, I genuitly dont know how it is handle in other code as I am unfamiliar with them.
Regarding the c and -c question, I implemented the converter to be consistent with @ye-luo’s comment.

[ye-luo]

Could you add converter tests under <tests/converter> for both cases?

[ye-luo]

With or without degeneracy. The total determinant count should remain the same.
In the case of 6_6 case, I saw 124 dets in both converter output.
However, this is not the case in 4_4 which doesn't use CSF. i don't feel the conversion in the DET case being right.

[NastaMauger]

@ye-luo
Yes, you are right.
I am confused about what the correct syntax is for the determinant case. In the CSF it is straightforward, but in the determinant representation, am I supposed to write these two degenerate groups as one?

<ci id="CIcoeff_3" coeff="0.10898" qc_coeff="0.10898" alpha="111111111111111111111111111001" beta="111111111111111111111111110110"/>
<ci id="CIcoeff_4" coeff="0.10898" qc_coeff="0.10898" alpha="111111111111111111111111110110" beta="111111111111111111111111111001"/>

I naively removed the ones that appeared more than once to avoid redundancy, but I am still confused about what the correct syntax should be.

[NastaMauger]

@ye-luo If I remove the determinant part and update the manual to state that the degeneracy of the CI works only for CSFs, would that help to merge this branch?

[ye-luo]

@ye-luo If I remove the determinant part and update the manual to state that the degeneracy of the CI works only for CSFs, would that help to merge this branch?

I'm sorry that I missed your precious reply. If you didn't hear a reply in a week, feel free to ping.
Please remove the WIP in the title. We usually don't actively review PRs with WIP which is mostly for the author to try out their change with CI.

@NastaMauger I sent you a slack message