Add GEOS coupling capability including heat and mass flux options in Icepack

[apcraig]

PR checklist

• Short (1 sentence) summary of your PR:
  Add GEOS coupling capability including heat and mass flux options in Icepack
• Developer(s):
  zhaobin74, apcraig
• Suggest PR reviewers from list in the column to the right.
• Please copy the PR test results link or provide a summary of testing completed below.
  standalone results are bit-for-bit, testing continues
• How much do the PR code changes differ from the unmodified code?
  • bit for bit except for some compilers due to compiler optimization
  • different at roundoff level
  • more substantial
• Does this PR create or have dependencies on Icepack or any other models?
  • Yes
  • No
• Does this PR update the Icepack submodule? If so, the Icepack submodule must point to a hash on Icepack's main branch.
  • Yes (to be done)
  • No
• Does this PR add any new test cases?
  • Yes
  • No
• Is the documentation being updated? ("Documentation" includes information on the wiki or in the .rst files from doc/source/, which are used to create the online technical docs at https://readthedocs.org/projects/cice-consortium-cice/. A test build of the technical docs will be performed as part of the PR testing.)
  • Yes
  • No, does the documentation need to be updated at a later time?
    • Yes
    • No
• Please document the changes in detail, including why the changes are made. This will become part of the PR commit log.

Add GEOS coupling updates. This allows coupling to the GEOS coupled system where a semi-implicit thermodynamic coupling scheme is introduced. Similar to the explicit case, the fields fsurfn are provided by the coupler, along with their derivatives with respect to surface temperature dfsurfn_dTs. In this case, calc_Tsfc is still set to true, allowing ice surface and internal temperature to be updated implicitly. The resultant surface temperature change is passed back to the atmosphere model via coupler to complete the full update of its temperature profiles. This middle-ground approach, enabled by semi_implicit_Tsfc=true, does not sacrifice accuracy because it does not need limiting effective conductivity as in the explicit case. In addition, in GEOS, the atmosphere model assumes vapor deposits or sublimates at 0 degC. In this case, mass conservation is enforced and the resulting discrepancy in energy is resolved by another term, de_vapor, and passed to ocean. This option is only on when vapor_flux_correction is true.

• Add 4 new shortwave terms, uvrdr, uvrdf, pardr, pardf to the coupling. These terms represent a breakdown of the direct and diffuse visible shortwave terms into two components, par = photosynthetical active radiation (400-700nm) and uvr = rest of the visible shortwave term (>700nm). The current visible shortwave is exactly represented by these two components. Includes adding atm forcing and terms associated with radiation passthru to the ocean.

• Add support for GEOS semi-implicit coupling of surface temperature. In GEOS, surface and latent heat flux is computed in the atmosphere at 0degC. The sea ice model has to respect that calculation, but then computes the d(dh)/dTs terms to correct the heatflux for the sea ice temperature which is then applied conservatively in the coupled system. Implementation includes turning off some of the heat flux calculations in Icepack. This is controlled by the semi_implicit_Tsfc namelist.

• Add calculation of a vapor flux correction. A correction is needed for GEOS coupling to compute a mass and enthalpy correction for evaporation and sublimation. This is controlled by the vapor_flux_correction namelist.

• Add mapl/geos coupling directory and coupling files

• Add opmask (orphan mask) for points that are NOT ocean/ice in the ocean/ice model but are ocean/ice in the atmosphere model. This allows for thermodyanamic calculations on the orphan points while not being involved in any sea ice dynamics. opmask determined by ocn_gridcell_frac which should be set by coupling layer at initialization.

• Add geosmom grid_type to read GEOS MOM grid files

• Add discover port

• Update documentation

• Change use of grid_type='tripole' where ns_boundary_type='tripole' is more appropriate.

• Add distribution_wght=blockfull option to move away from CPP in init_domain_blocks. This option turns off land block elimination and sets all blocks to maximum weight for distribution.

• Clean up some declarations in ice_flux.F90 and ice_state.F90

• Add some new initialization output for new GEOS options and to clean up grid_type output

[apcraig]

This supercedes #959 and is consistent with CICE-Consortium/Icepack#518. The Icepack PR needs to be completed and added to this PR. Some questions/comments,

• I have used geos_heatflux and geos_massflux as the initial namelist/parameter variables. But I think these should be named without geos and should represent a namelist like "coupling_associated_with_atmice_heat_flux_calculated_in_the_atmosphere_at_0degC_with_CICE_computing_d(hf)/dTs". Thoughts? Should we have 1 namelist variable to turn this on or are 2 namelist more appropriate?

• Are there standalone CICE tests we should or could do with these new features? Or is this a coupling-only option?

• I have NOT implemented the changes to the initialization interfaces to set the pes, block_size, initial time, or dt from the coupling layer. These should all be settable by the namelist. Is there a reason the namelist settings are not adequate? For most models, these are known "before run time" and can be set in the namelist automatically via scripts. If some of these things are only known at run-time, let me know and we can add the feature.

• There is a new namelist option, distribution_wght='blockfull' which will turn off land block elimination and avoid a CPP. This option should be used for GEOS.

• I have used ocn_gridcell_frac instead of "frocean".

• I have used grid_type='geosmom' as the option to read the GEOS grid file.

[apcraig]

@zhaobin74. Please checkout the latest Icepack and CICE GEOS PRs and test in GEOS. I made a couple minor changes in the mapl/geos files, but just for variable name changes. Those subroutines could be slightly out of date with respect to the current versions, please check. I'm sure we will need to iterate a bit to get this all working, happy to do that. Please let me know if there are any questions about how to check the code or PR changes back to my branches via git.

[zhaobin74]

Thank you, @apcraig, for doing this great work.

I'll check out your branch and make some test runs within GEOS. We recently got a Data Atmosphere working, which makes testing coupled ocean/seaice more convenient. My goal is to achieve a zero diff between GEOS branch and yours.

[apcraig]

Thanks @zhaobin74, keep me posted. I'm sure we'll have to iterate a few times, just let me know what needs to change or feel free to PR some changes to my branch if you prefer to work that way. I'm hoping we can converge fairly quickly.

[zhaobin74]

@apcraig, I have made some progress. I made changes to mapl/geos/CICE_InitMod.F90 to make it work with your mods. It is still not working(crash) but how should I get these updates to your branch?

[apcraig]

@zhaobin74 The best way to get the changes onto my PR would be to PR to it. To do so, you would clone my branch then push that branch onto your fork. You can make the changes in your fork and then PR those changes back to my branch. If you need more specific details, let me know. Alternatively, you could email me your changes as attached files or send me the git diffs in an email (if they are fairly minor), and I can incorporate them into my branch and update the PR. Let me know how I can help.

[zhaobin74]

@zhaobin74 The best way to get the changes onto my PR would be to PR to it. To do so, you would clone my branch then push that branch onto your fork. You can make the changes in your fork and then PR those changes back to my branch. If you need more specific details, let me know. Alternatively, you could email me your changes as attached files or send me the git diffs in an email (if they are fairly minor), and I can incorporate them into my branch and update the PR. Let me know how I can help.

Thanks, @apcraig. I'll fork and make PR to your branch. We may need go through several iterations and Pr's work better.

[apcraig]

@zhaobin74, sounds great. I expect we'll need to do a few iterations to converge, but I don't anticipate any significant problems. Thanks!

[zhaobin74]

@apcraig, please see my first round of bug fixes. At lease it compiled and ran for a few time steps in GEOS without crashing. More to come.

[eclare108213]

@zhaobin74 what is the status of this project/PR? Is the related Icepack PR ready, or do you think there will still be changes to it as you're working on this CICE portion?

[zhaobin74]

@zhaobin74 what is the status of this project/PR? Is the related Icepack PR ready, or do you think there will still be changes to it as you're working on this CICE portion?

Hi @eclare108213, sorry for the delay of this PR. I have been at leave for some time and just catching up. The PR is almost finished, I think. I have one more update to @apcraig's branch. Once it is done, it can be merged.

[apcraig]

@zhaobin74, I just merged your latest changes into CICE. Do you feel both the Icepack and CICE PRs are OK from your side? If so, I will merge main to the branch and test all the changes again in standalone Icepack and CICE and we can continue the merge process.

@eclare108213, @zhaobin74 please also have a look at CICE-Consortium/Icepack#518. There are still several open questions which we need to address together. @zhaobin74, can you reply to some of @eclare108213's comments there. I am happy to update the codes as needed once I have some guidance and/or changes can be PR'ed from others. There are several questions about documentation and implementation.

[zhaobin74]

@zhaobin74, I just merged your latest changes into CICE. Do you feel both the Icepack and CICE PRs are OK from your side? If so, I will merge main to the branch and test all the changes again in standalone Icepack and CICE and we can continue the merge process.

@eclare108213, @zhaobin74 please also have a look at CICE-Consortium/Icepack#518. There are still several open questions which we need to address together. @zhaobin74, can you reply to some of @eclare108213's comments there. I am happy to update the codes as needed once I have some guidance and/or changes can be PR'ed from others. There are several questions about documentation and implementation.

@apcraig, thanks for merging the PR. I think both PRs are ok for merging with the main after more testing in standalone mode. ( I did that for my PR and got 0-diff).

I will address some of the remaining questions.

[apcraig]

Just FYI, I am rebasing this branch from main. main has some fairly significant modifications to the grid implementation, so there will be several modifications related to the geos grid that will need to be tested. I will be doing a force push due to the rebase on both Icepack and CICE due to conflicts which means old sandboxes will probably not be usable anymore. You'll want to checkout the latest code after my rebase and force push.

[apcraig]

I have rebased the geos branches to main and force pushed the Icepack and CICE geos changes. These are reflected in the PRs. Please review the PRs again, check them out new (you cannot merge into existing sandboxes because of the force push), and do some testing. As I mentioned above, the grid implementation has changed. In particular, setting the geos grid in CICE is now done with

grid_format = 'geosnc'

I have run tests and am seeing answer changes on Cray (but not intel or gnu) that I suspect are just compiler related. I will confirm. Lets continue to try to move forward. Thanks!

[zhaobin74]

I have rebased the geos branches to main and force pushed the Icepack and CICE geos changes. These are reflected in the PRs. Please review the PRs again, check them out new (you cannot merge into existing sandboxes because of the force push), and do some testing. As I mentioned above, the grid implementation has changed. In particular, setting the geos grid in CICE is now done with

grid_format = 'geosnc'

I have run tests and am seeing answer changes on Cray (but not intel or gnu) that I suspect are just compiler related. I will confirm. Lets continue to try to move forward. Thanks!

Sounds good, @apcraig. I'll do a test with a fresh checkout of geos branch.
One question: if using grid_format = 'geosnc' , what should be set to grid_type?
Thank you.

[apcraig]

Sounds good, @apcraig. I'll do a test with a fresh checkout of geos branch.
One question: if using grid_format = 'geosnc' , what should be set to grid_type?

Good question. I suggest you use "displaced_pole". The grid_format has become somewhat redundant with grid_format, ew_boundary_type, and ns_boundary_type really playing a larger role in setting up the grid and the boundary conditions for the grid now.

[zhaobin74]

@apcraig, sorry I have to make another PR with some fixes. My previous PR missed some variable defines. I also have to fix some GEOS driver routines due to recent icepack interface change.

[zhaobin74]

Sounds good, @apcraig. I'll do a test with a fresh checkout of geos branch.
One question: if using grid_format = 'geosnc' , what should be set to grid_type?

Good question. I suggest you use "displaced_pole". The grid_format has become somewhat redundant with grid_format, ew_boundary_type, and ns_boundary_type really playing a larger role in setting up the grid and the boundary conditions for the grid now.

@apcraig, I had a look at the new ice_grid module and noticed the newly introduced mom super grid format mom_nc. I am wondering if geosnc can be consolidated with mom_nc. GEOS coupled model grid is essentially MOM tripolar grid, with a slightly different land sea mask due to orphan points. Right now we extract the grid information from MOM grid spec and put them into a pop grid format for CICE to digest. If CICE can directly read in MOM's grid, we can adopt that for GEOS too. Any thoughts? Thanks.

[apcraig]

@apcraig, I had a look at the new ice_grid module and noticed the newly introduced mom super grid format mom_nc. I am wondering if geosnc can be consolidated with mom_nc. GEOS coupled model grid is essentially MOM tripolar grid, with a slightly different land sea mask due to orphan points. Right now we extract the grid information from MOM grid spec and put them into a pop grid format for CICE to digest. If CICE can directly read in MOM's grid, we can adopt that for GEOS too. Any thoughts? Thanks.

I think however you want to proceed is fine for us. We can support your GEOS grid file converted from MOM via your current implementation. We can also support reading in your MOM files directly if that works for you. It would be great if we could use the new MOM method for your MOM files, but we could also either modify or create a new grid reading method if needed. CICE is comfortable supporting multiple grid reading methods that work best for the community, and we can support multiple methods for GEOS if that's also useful. Just let us know what you prefer to do.

[zhaobin74]

@apcraig, I had a look at the new ice_grid module and noticed the newly introduced mom super grid format mom_nc. I am wondering if geosnc can be consolidated with mom_nc. GEOS coupled model grid is essentially MOM tripolar grid, with a slightly different land sea mask due to orphan points. Right now we extract the grid information from MOM grid spec and put them into a pop grid format for CICE to digest. If CICE can directly read in MOM's grid, we can adopt that for GEOS too. Any thoughts? Thanks.

I think however you want to proceed is fine for us. We can support your GEOS grid file converted from MOM via your current implementation. We can also support reading in your MOM files directly if that works for you. It would be great if we could use the new MOM method for your MOM files, but we could also either modify or create a new grid reading method if needed. CICE is comfortable supporting multiple grid reading methods that work best for the community, and we can support multiple methods for GEOS if that's also useful. Just let us know what you prefer to do.

@apcraig, maybe that could be another PR down the road. Let's wrap up this one. My tests show the current head is working ok.

[apcraig]

@zhaobin74 I agree, we can easily update the grid reading methods later.

So, just to be clear. Do you feel the current changes to Icepack and CICE are ready to merge to the trunk? If so, I'll do some final testing on the standalone models and then will merge the PRs. If we need to iterate further or if you want to do more testing in GEOS, please let me know, that's fine too.

[zhaobin74]

@zhaobin74 I agree, we can easily update the grid reading methods later.

So, just to be clear. Do you feel the current changes to Icepack and CICE are ready to merge to the trunk? If so, I'll do some final testing on the standalone models and then will merge the PRs. If we need to iterate further or if you want to do more testing in GEOS, please let me know, that's fine too.

@apcraig, I think it is ready to merge back to the main. One final issue has not been resolved, but right now I am inclined to partition the rain into open ocean and ice covered parts such that this if block is not needed.

Should any issue pops up later, we could fix them, and it is going to be much smaller scale. Thanks.

[apcraig]

As far as I can tell, the Icepack and CICE PRs are ready for the GEOS modifications. Could everyone do a final review and approve. I will also do a final suite of testing. There are a couple extra caveats with respect to merging onto main. Derecho will be down next week, and we are planning to release very soon, so we'll have to see how the timing/review/testing works out and whether this will be part of the release or not. If anyone has a particular opinion about that, please say so.

[apcraig]

I completed a full suite of testing on derecho, tests look fine. Some answer changes with some compilers, but this seems to be due to compiler optimization. I tested a few cases manually with the debug flags and those are bit-for-bit. I think this could be ready to merge after final review. https://github.com/CICE-Consortium/Test-Results/wiki/icepack_by_hash_forks#be5757df4ffa79e6bd283445892ce5c31c054780, https://github.com/CICE-Consortium/Test-Results/wiki/cice_by_hash_forks#c98b212136e9a716c4d89212f5459fcc4ee9453b

[apcraig]

I am running a quick_suite testsuite on derecho now and will wait for the GHActions to complete. Please approve or let me know if there are other issues.