Should one add Band Structure and Jij's workchains?

[JPchico]

I was looking at the documentation on how to submit Band structure calculations and Jij's determinations from previous calculations. And I have a couple of questions

1. Should one generate band structure and/or Jij's workchains?: The calculations themselves seem quite straightforward once one has an output from an SCF or Voronoi calculation. Specially the band structure since one could in principle just use the DOS workchain with modified inputs for the submission. In that case, I think that adding some examples of this might be a good idea.

2. Should one modify the workchains in such a way as to allow the direct calculation of lets say the DOS, without a previous calculation? I.e. the DOS calculation, for example, would search if the remote is present, if it is everything proceeds as normal, otherwise, voronoi, and SCF (if required) calculations are performed. What do you think?

3. Not so related to this, should one add a verbose command, to limit the printing of some of the printout data to stdout?

[broeder-j]

1. I know that @PhilippRue does calculate Jijs, if it is possible and straight forward with what is there, it should be enough to add some examples. @JPchico we are currently implementing some functions to provide input dicts for workchains and in addition to expose them on the command line, or the same workflow in different ways. One can surely do this for different 'run modes', that would make it also user friendlier.

2. I cannot speak for @PhilippRue, but in principle there is nothing against making these workchains more 'turn-key'. But someone has to do it ;-) and the complexer a workchain becomes the higher the failure rate goes if one runs the complex path. If you simulate many systems I would advice you to first run an SCF, and then if these are turnout to be fine run further workchains.

To 3. you can change aiida loglevels, which will in the end determine what details make it into the log/report and which are also shown if you run a workchain. If you go to loglevel critical, all messages should disappear, if not we have to implement it right.
When using the command line interface one could temporary change the loglevel to make it more verbose if wanted.

[broeder-j]

add on to 3. the process report has is own loglevel in aiida and is always the same. see https://github.com/aiidateam/aiida-core/blob/develop/aiida/common/log.py
and https://github.com/aiidateam/aiida-core/blob/develop/aiida/engine/processes/process.py
therefore currently it is not straight forward to reduce the verbosity and I am not sure how one would enable logging with different log levels to the report. Maybe by calling self.logger instead of self.report one could stay with different levels of verbosity in the report

[PhilippRue]

1. First I would like to say that @RubelMozumder is already working on making a workflow for the bandstructure calculation. I think this will be merged in soon. The current design for this workflow is analogous to the DOS workflow.
   I believe that the Jij calculation could be made into a similar workflow since you always set a certain set of parameters, which is not difficult but cumbersome. @JPchico if you like to implement a Jij workchain, feel free to do so ;)

2. As far as turn-key solutions are concerned, I often use the scf workchain with the check_dos mode which calculates the DOS at different stages of the calculation (i.e. from the voronoi starting potential and after self-consistency). This is mainly to cross check the result of the calculation. Only then I usually use the DOS workflow to calculate the DOS in higher accuracy and for the energy range I am most interested in. We probably have to improve on the robustness of the scf workchain to have a real benefit from allowing the DOS workchain to start a scf if the input is not a previous calculation.

3. I agree that we should limit the output to stdout, most of the output is left over from development anyways and can therefore be suppressed in most cases. We probably have to change some things to be able to control this with the loglevel @broeder-j mentioned.

[JPchico]

Thanks @broeder-j for your answers!

1. That is very nice to hear. Since it can be confusing for users otherwise. I do not mind adding some examples after I'm happy with the results that I obtain from my own tests for these more advanced workchains.

2. Absolutely, I agree that it is way more complicated, I have built some for other workchains for other codes (not yet public, working on it) and the turn key approach is indeed much more complicated. I think that if these test works well and KKR can be used by more people ( I have around 4-5 users that might do simulations with it) it might be feasible for me to dedicate more time for these workchains, and I could work in these workchains. I will know more in the next comming weeks.

3. Okay that is pretty good to know, I was just thinking in an Bool kind of input like what the QE and VASP plugins have done, in which they have an explicit verbose input and they gate some of the stdout to this input.

[JPchico]

1. First I would like to say that @RubelMozumder is already working on making a workflow for the bandstructure calculation. I think this will be merged in soon. The current design for this workflow is analogous to the DOS workflow.

Fantastic! That is very nice to hear.

I believe that the Jij calculation could be made into a similar workflow since you always set a certain set of parameters, which is not difficult but cumbersome. @JPchico if you like to implement a Jij workchain, feel free to do so ;)

Will probably do so then ;)

2. As far as turn-key solutions are concerned, I often use the scf workchain with the check_dos mode which calculates the DOS at different stages of the calculation (i.e. from the voronoi starting potential and after self-consistency). This is mainly to cross check the result of the calculation. Only then I usually use the DOS workflow to calculate the DOS in higher accuracy and for the energy range I am most interested in. We probably have to improve on the robustness of the scf workchain to have a real benefit from allowing the DOS workchain to start a scf if the input is not a previous calculation.

Sure, I saw many comments in the code of things that you guys were thinking of doing. I wonder though, why the workchain does not use the BaseRestartWorkchain? Is it because it would be too difficult to control the fact that one needs to use voro_start?

[PhilippRue]

Sure, I saw many comments in the code of things that you guys were thinking of doing. I wonder though, why the workchain does not use the BaseRestartWorkchain? Is it because it would be too difficult to control the fact that one needs to use voro_start?

This is simply because I never finished implementing this xD I wanted to let BaseRestartWorkchain take care of all 'computer problems' like out of memory issues etc. Then instead of submitting the calculation inside the workchains you should be able to use the base workchain. But since the current structure works finishing this was never pushed to the top of my to do list ;)

[PhilippRue]

3. Okay that is pretty good to know, I was just thinking in an Bool kind of input like what the QE and VASP plugins have done, in which they have an explicit verbose input and they gate some of the stdout to this input.

Yes I think that is the easiest solution.

[JPchico]

Sure, I saw many comments in the code of things that you guys were thinking of doing. I wonder though, why the workchain does not use the BaseRestartWorkchain? Is it because it would be too difficult to control the fact that one needs to use voro_start?

This is simply because I never finished implementing this xD I wanted to let BaseRestartWorkchain take care of all 'computer problems' like out of memory issues etc. Then instead of submitting the calculation inside the workchains you should be able to use the base workchain. But since the current structure works finishing this was never pushed to the top of my to do list ;)

Okay xD I mean it works so it makes no sense to break it. But I wonder if perhaps it would be good to make a branch, where one makes a base_kkr and base_voro workchains, which use the BaseRestartWorkchain these could then be called by voro_start and kkr_scf such that one can have some of the automatic error handling coming from there. I think the only question there is how to pass the potential, I guess the parent_folder approach as used in the kkr calculation could work without problem. It is something I could have a little stab at. If that is okay by you guys

[broeder-j]

3. Okay that is pretty good to know, I was just thinking in an Bool kind of input like what the QE and VASP plugins have done, in which they have an explicit verbose input and they gate some of the stdout to this input.

could you point me to it? I am not sure what you mean.

[JPchico]

3. Okay that is pretty good to know, I was just thinking in an Bool kind of input like what the QE and VASP plugins have done, in which they have an explicit verbose input and they gate some of the stdout to this input.

could you point me to it? I am not sure what you mean.

Sure thing, you can look at this example in the aiida-vasp they use the verbose input in the workchain.

[PhilippRue]

Okay xD I mean it works so it makes no sense to break it. But I wonder if perhaps it would be good to make a branch, where one makes a base_kkr and base_voro workchains, which use the BaseRestartWorkchain these could then be called by voro_start and kkr_scf such that one can have some of the automatic error handling coming from there. I think the only question there is how to pass the potential, I guess the parent_folder approach as used in the kkr calculation could work without problem. It is something I could have a little stab at. If that is okay by you guys

Sure, go ahead.

[broeder-j]

Sure thing, you can look at this example in the aiida-vasp they use the verbose input in the workchain.

Thanks. Yes. this is indeed a good idea. I would maybe make it an integer instead of a boolean, to control the verbosity like with usual python logging. That way one could also leave for example debug massages in the workchain code, which one can switch on during development.

[JPchico]

@PhilippRue and @broeder-j sorry I had not had time to get into this. However, I'm doing some KKR simulations again, and I have had time to look into the workflow. Something I was doing (I'll be making a PR regarding issue #61 ) is trying to get the format to follow the pre-commit hooks that are being followed in aiida-core. I think this is a positive thing as it should make the maintenance easier. It might be good to add it as a github action. However, I think that one needs to be very sure of that before messing around with it too much.

[PhilippRue]

Moved open issue of verbosity control to issue #108