Merge pull request #972 from iiasa/enh/model-module

Separate models.py into submodules

Author: khaeru

RELEASE_NOTES.rst

@@ -1,6 +1,24 @@
 Next release
 ============
 
+Migration notes
+---------------
+
+Adjust any imports like the following:
+
+.. code-block:: python
+
+    from message_ix.models import DIMS, Item, MACRO, MESSAGE, MESSAGE_MACRO
+
+…to:
+
+.. code-block:: python
+
+    from message_ix.common import DIMS, Item
+    from message_ix.macro import MACRO
+    from message_ix.message import MESSAGE
+    from message_ix.message_macro import MESSAGE_MACRO
+
 All changes
 -----------
 
@@ -20,6 +38,13 @@ All changes
   - Revise :ref:`equation_commodity_balance_aux` to include input and output flows based on |CAP| and |CAP_NEW|.
   - New :class:`.MESSAGE` / :meth:`.Scenario.solve` option :py:`cap_comm=True` to enable this representation.
 
+- The former module :py:`message_ix.models` is split to distinct submodules (:pull:`972`):
+
+  - :mod:`message_ix.common` includes :class:`.GAMSModel` and related code.
+  - :mod:`message_ix.macro` includes :class:`.MACRO`.
+  - :mod:`message_ix.message` includes :class:`.MESSAGE`.
+  - :mod:`message_ix.message_macro` includes :class:`.MESSAGE_MACRO`.
+
 - Document the :ref:`minimum version of Java <install-java>` required for :class:`ixmp.JDBCBackend <ixmp.backend.jdbc.JDBCBackend>` (:pull:`962`).
 - Improve type hinting (:pull:`963`).
 
@@ -61,7 +86,7 @@ Users **should**:
 All changes
 -----------
 
-- Some MESSAGEix :doc:`tutorials <tutorials>` are runnable with the :class:`.IXMP4Backend` introduced in :mod:`ixmp` version 3.11 (:pull:`894`, :pull:`941`).
+- Some MESSAGEix :doc:`tutorials <tutorials>` are runnable with the :class:`~ixmp.backend.ixmp4.IXMP4Backend` introduced in :mod:`ixmp` version 3.11 (:pull:`894`, :pull:`941`).
   See `Support roadmap for ixmp4 <https://github.com/iiasa/message_ix/discussions/939>`__ for details.
 - Add the :py:`concurrent=...` model option to :class:`.MACRO` (:pull:`808`).
 - Adjust use of :ref:`type_tec <mapping-sets>` in :ref:`equation_emission_equivalence` (:pull:`930`, :issue:`929`, :pull:`935`).
@@ -174,7 +199,7 @@ Migration notes
   NOTE: this may result in changes to the solution.
   In order to use the previous default `lpmethod`, the user-specific default setting can be set through the user's ixmp configuration file.
   Alternatively, the `lpmethod` can be specified directly as an argument when solving a scenario.
-  Both of these configuration methods are further explained :meth:`here <message_ix.models.GAMSModel>`.
+  Both of these configuration methods are further documented at :class:`.GAMSModel`.
 
 - The dimensionality of one set and two parameters (``map_tec_storage``, ``storage_initial``, and ``storage_self_discharge``) are extended to allow repesentation of the mode of operation of storage technologies and the temporal level of storage containers.
   If these items are already populated with data in a Scenario, this data will be incompatible with the MESSAGE GAMS implementation in this release; a :class:`UserWarning` will be emitted when the :class:`.Scenario` is instantiated, and :meth:`~.message_ix.Scenario.solve` will raise a :class:`ValueError`.
@@ -421,7 +446,7 @@ All changes
 - :pull:`281`: Test and improve logic of :meth:`.years_active` and :meth:`.vintage_and_active_years`.
 - :pull:`269`: Enforce ``year``-indexed columns as integers.
 - :pull:`256`: Update to use :obj:`ixmp.config` and improve CLI.
-- :pull:`255`: Add :mod:`message_ix.testing.nightly` and ``message-ix nightly`` CLI command group for slow-running tests.
+- :pull:`255`: Add :py:`message_ix.testing.nightly` and ``message-ix nightly`` CLI command group for slow-running tests.
 - :pull:`249`, :pull:`259`: Build MESSAGE and MESSAGE_MACRO classes on ixmp model API; adjust Scenario.
 - :pull:`235`: Add a reporting tutorial.
 - :pull:`236`, :pull:`242`, :pull:`263`: Enhance reporting.

doc/api.rst

@@ -129,17 +129,21 @@ The full API is also available from R; see :doc:`rmessageix`.
 Model classes
 -------------
 
-.. currentmodule:: message_ix.models
+.. currentmodule:: message_ix
 
 .. autosummary::
 
-   MESSAGE
-   MACRO
-   MESSAGE_MACRO
-   GAMSModel
-   DEFAULT_CPLEX_OPTIONS
-   Item
-   ItemType
+   ~message.MESSAGE
+   ~macro.MACRO
+   ~message_macro.MESSAGE_MACRO
+   ~common.GAMSModel
+   ~common.DEFAULT_CPLEX_OPTIONS
+   ~common.Item
+   ~ixmp.backend.ItemType
+
+.. currentmodule:: message_ix.common
+
+.. automodule:: message_ix.common
 
 .. autodata:: DEFAULT_CPLEX_OPTIONS
 
@@ -149,7 +153,8 @@ Model classes
    :members:
    :exclude-members: defaults
 
-   The :class:`.MESSAGE`, :class:`MACRO`, and :class:`MESSAGE_MACRO` child classes encapsulate the GAMS code for the core MESSAGE (or MACRO) mathematical formulation.
+   The :class:`.MESSAGE`, :class:`.MACRO`, and :class:`.MESSAGE_MACRO` child classes
+   encapsulate the GAMS code for the core MESSAGE (or MACRO) mathematical formulation.
 
    The class receives `model_options` via :meth:`.Scenario.solve`. Some of these are passed on to the parent class :class:`ixmp.model.gams.GAMSModel` (see there for a list); others are handled as described below.
 
@@ -244,6 +249,10 @@ Model classes
       * - **var_list**
         - :obj:`None`
 
+.. currentmodule:: message_ix.message
+
+.. automodule:: message_ix.message
+
 .. autoclass:: MESSAGE
    :members: initialize
    :exclude-members: defaults
@@ -267,6 +276,10 @@ Model classes
       Keys are the names of items (sets, parameters, variables, and equations); values are :class:`.Item` instances.
       These include all items listed in the MESSAGE mathematical specification, i.e. :ref:`sets_maps_def` and :ref:`parameter_def`.
 
+.. currentmodule:: message_ix.macro
+
+.. automodule:: message_ix.macro
+
 .. autoclass:: MACRO
    :members:
    :exclude-members: items
@@ -283,6 +296,10 @@ Model classes
    .. autoattribute:: items
       :no-value:
 
+.. currentmodule:: message_ix.message_macro
+
+.. automodule:: message_ix.message_macro
+
 .. autoclass:: MESSAGE_MACRO
    :members:
    :exclude-members: items
@@ -307,12 +324,10 @@ Model classes
    .. autoattribute:: items
       :no-value:
 
-.. autodata:: DIMS
-.. autoclass:: Item
+.. autodata:: message_ix.common.DIMS
+.. autoclass:: message_ix.common.Item
    :members:
 
-.. currentmodule:: message_ix.macro
-
 .. _utils:
 
 Utility methods
@@ -321,14 +336,8 @@ Utility methods
 .. automodule:: message_ix.util
    :members: expand_dims, copy_model, make_df
 
-.. automodule:: message_ix.util.sankey
-   :members: map_for_sankey
-
 Testing utilities
 -----------------
 
 .. automodule:: message_ix.testing
    :members: make_austria, make_dantzig, make_westeros, tmp_model_dir
-
-.. automodule:: message_ix.testing.nightly
-   :members:

doc/conf.py

@@ -165,6 +165,7 @@
     # top-level module in objects.inv. Resolve these using :doc:`index` or similar for
     # each project.
     "dask$": ":std:doc:`dask:index`",
+    "jpype$": ":std:doc:`jpype:index`",
     "plotnine$": ":class:`plotnine.ggplot`",
 }
 

doc/install-adv.rst

@@ -230,7 +230,7 @@ This implies five of the six available groups of extra requirements:
 
 - ``docs`` includes packages required to build this documentation locally,
   including ``message_ix[report]`` and all *its* requirements,
-- ``ixmp4`` includes packages require to use :class:`ixmp.IXMP4Backend <.IXMP4Backend>`,
+- ``ixmp4`` includes packages require to use :class:`ixmp.IXMP4Backend <ixmp.backend.ixmp4.IXMP4Backend>`,
 - ``report`` includes packages required to use the built-in :doc:`reporting <reporting>` features of :mod:`message_ix`,
 - ``sankey`` includes packages required to use :meth:`.Reporter.add_sankey`,
 - ``tests`` includes packages required to run the test suite,

doc/macro.rst

@@ -73,10 +73,12 @@ The required dimensions or symbol of each item are given in the same notation us
 
 - ``price_ref`` (:math:`n, s`): prices of MACRO sector output in the reference year.
   These can be constructed from the MESSAGE variable ``PRICE_COMMODITY``, using the ``config`` mapping.
-  If not provided, :mod:`message_ix.macro` will identify the reference year and extrapolate reference values using an exponential function fitted to ``PRICE_COMMODITY`` values; see :func:`.macro.extrapolate`.
+  If not provided, :mod:`message_ix.macro.calibrate` will identify the reference year
+  and extrapolate reference values using an exponential function fitted to ``PRICE_COMMODITY`` values.
+  See :func:`~.macro.calibrate.extrapolate`.
 - ``cost_ref`` (:math:`n`): total cost of the energy system in the reference year.
   These can be constructed from the MESSAGE variable ``COST_NODAL_NET``, including dividing by a factor of 1000.
-  If not provided, :mod:`message_ix.macro` will extrapolate using :func:`.macro.extrapolate`.
+  If not provided, :mod:`message_ix.macro.calibrate` will extrapolate using :func:`~.macro.calibrate.extrapolate`.
 - ``demand_ref`` (:math:`n, s`): demand for MACRO sector output in the reference year.
 - ``lotol`` (:math:`n`): tolerance factor for lower bounds on MACRO variables.
 - ``esub`` (:math:`\epsilon_n`): elasticity of substitution between capital-labor and energy.
@@ -228,9 +230,9 @@ Alternatively the arguments can be specified either in :file:`models.py`.
 Code documentation
 ==================
 
-.. currentmodule:: message_ix.macro
+.. currentmodule:: message_ix.macro.calibrate
 
-.. automodule:: message_ix.macro
+.. automodule:: message_ix.macro.calibrate
    :members:
 
    The functions :func:`add_model_data` and :func:`calibrate` are used by :meth:`.Scenario.add_macro`.

doc/reporting.rst

@@ -179,7 +179,7 @@ These include:
 
 Other added keys include:
 
-- :mod:`message_ix` adds the standard short symbols for |MESSAGEix| dimensions (sets) based on :data:`.models.DIMS`.
+- :mod:`message_ix` adds the standard short symbols for |MESSAGEix| dimensions (sets) based on :data:`.common.DIMS`.
   Each of these is also available in a Reporter: for example :py:`rep.get("n")` returns a list with the elements of the |MESSAGEix| set named "node";
   :py:`rep.get("t")` returns the elements of the set "technology", and so on.
   These keys can be used as input to other computations.

message_ix/__init__.py

@@ -8,7 +8,9 @@
 from ixmp.util import DeprecatedPathFinder
 
 from .core import Scenario
-from .models import MACRO, MESSAGE, MESSAGE_MACRO
+from .macro import MACRO
+from .message import MESSAGE
+from .message_macro import MESSAGE_MACRO
 from .report import Reporter
 from .util import make_df