Test against ixmp4

[khaeru]

This began as a minimal PR to check that the message_ix test suite runs using ixmp4 storage (i.e. without the ixmp.JDBCBackend and underlying Java code).

It was then substantially expanded by @glatterf42 such that (most of) the message_ix test suite and all of the tutorials are confirmed to run using the IXMP4Backend.

Some things not in scope for this PR:

• Changes to the behaviour/API of message_ix.Scenario.
• Migrating large chunks of the ixmp code (including its test suite) to message_ix.
• User-facing documentation explaining the changes and how to use the new backend with message_ix; this will be handled by Support roadmap for `ixmp4` #939 and a follow-up PR.

Details

In order to achieve the testing, it:

• Installs from the branch for Implement ixmp4 shim ixmp#552 the ixmp main branch that:
  • Provides a minimal 'shim' layer between the behaviour of ixmp.Platform, ixmp.TimeSeries, ixmp.Scenario and the underlying ixmp4 API.
  • Ensures that all of the behaviour of those 3 classes continues to pass the ixmp test suite when using ixmp4 storage. (message_ix and downstream/user code rely on that behaviour.)
  • Will (but does not yet, as of 2024-11-28) parametrize the ixmp.testing.test_mp fixture to yield either the current ixmp.JDBCBackend-backed test Platform, or an ixmp4-backed test Platform.
• Confirms all of the behaviour of message_ix.Scenario continues to pass the message_ix.tests suite, with either storage option. (message-ix-models and downstream/user code rely on this behaviour.)

The branch is used by iiasa/message-ix-models#257.

Implementation/to dos:

• As a consequence of parametrizing ixmp.testing.test_mp, all tests in message_ix.tests that use the test_mp fixture will run twice: once with JDBCBackend, and once with ixmp4.
• Some other tests may not be parametrized automatically by the upstream changes:
  • Identify these.
  • Parametrize them manually.
• Rebase this branch frequently.

How to review

• Read the diff.
• Note the CI checks all pass.

PR checklist

• Merge upstream PRs including Implement ixmp4 shim ixmp#552, Adjust for message-ix test suite and tutorials ixmp#575.
• Continuous integration checks all ✅
• Add or expand tests; coverage checks both ✅
• Add, expand, or update documentation. See above.
• Update release notes.

[codecov]

Codecov Report

❌ Patch coverage is 66.09272% with 256 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.6%. Comparing base (3b896d3) to head (a191e05).
⚠️ Report is 87 commits behind head on main.

Files with missing lines	Patch %	Lines
message_ix/util/scenario_setup.py	21.8%	104 Missing ⚠️
message_ix/util/gams_io.py	14.7%	81 Missing ⚠️
message_ix/tests/util/test_gams_io.py	28.8%	32 Missing ⚠️
message_ix/tests/test_tutorials.py	60.0%	10 Missing ⚠️
message_ix/core.py	67.8%	9 Missing ⚠️
message_ix/util/ixmp4.py	52.6%	9 Missing ⚠️
message_ix/models.py	86.6%	8 Missing ⚠️
message_ix/tests/test_report.py	88.8%	2 Missing ⚠️
message_ix/tests/test_core.py	97.9%	1 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (3b896d3) and HEAD (a191e05). Click for more details.

HEAD has 227 uploads less than BASE

Flag	BASE (3b896d3)	HEAD (a191e05)
	233	6

Additional details and impacted files

@@           Coverage Diff           @@
##            main    #894     +/-   ##
=======================================
- Coverage   95.9%   90.6%   -5.4%     
=======================================
  Files         48      53      +5     
  Lines       4487    4988    +501     
=======================================
+ Hits        4305    4521    +216     
- Misses       182     467    +285     

Files with missing lines	Coverage Δ
message_ix/testing/__init__.py	99.7% <100.0%> (+<0.1%)	⬆️
message_ix/tests/model/test_message.py	100.0% <100.0%> (ø)
message_ix/tests/report/test_operator.py	100.0% <100.0%> (ø)
message_ix/tests/test_cli.py	91.4% <100.0%> (+0.8%)	⬆️
message_ix/tests/test_feature_addon.py	100.0% <100.0%> (ø)
...age_ix/tests/test_feature_bound_activity_shares.py	100.0% <100.0%> (ø)
message_ix/tests/test_feature_bound_emission.py	100.0% <100.0%> (ø)
message_ix/tests/test_feature_capacity_factor.py	100.0% <100.0%> (ø)
message_ix/tests/test_feature_duration_time.py	100.0% <100.0%> (ø)
message_ix/tests/test_feature_price_commodity.py	100.0% <100.0%> (ø)
... and 27 more

... and 3 files with indirect coverage changes

[khaeru]

At iiasa/ixmp#557, @glatterf42 asked:

For example: ixmp4.Run.clone() doesn't support first_model_year […]. How crucial are these features?

To expand on iiasa/ixmp#557 (comment): I briefly played around with #873 last year. Via that PR or something else, there are multiple possible solutions. I'm not sure right now which achieves our goal of minimizing breaking changes:

1. For one message_ix.Scenario.clone() can check if it's talking to JDBCBackend or IXMP4Backend:
   • If JDBCBackend, it passes on the first_model_year argument to the Java code, preserving the current behaviour exactly.
   • If IXMP4Backend, it asks for a simple clone, then invokes shift_period() to modify the data.
2. Another possibility is that message_ix.Scenario.clone() never passes down a first_model_year parameter—even to JDBCBackend—and always invokes shift_period() itself.

While (1) is not 100% 'clean' in the sense that message_ix.Scenario shouldn't need to know or care what backend it is talking to, the limitations of the Java code (see the other PR) might make (2) unworkable, in which case (1) would be, I think, the simplest compromise option.