Model selection with PEtab Select

Model selection with PEtab Select

Introduction

PEtab Select is a Python package that can be used to perform model selection in d2d. The model selection problem must be specified by PEtab Select files which are based on the PEtab file format for exchanging parameter estimation problems. The d2d support for PEtab files is decribed here.

Installation

To use PEtab Select with d2d, we use Matlab's Python environment for calling Python functions directly in Matlab. In the following we give a step-by-step guide for installation. More details can be found here.

Requirements

• Python >= 3.10
• Matlab >= R2022b (see list of compatible Matlab and Python versions)
• Current d2d installation

Short installation guide

If you already have Python, Matlab and d2d installed, run the following commands to setup PEtab Select with d2d:

1. Open a system terminal and install the petab_select Python module (including the plot package):
   pip install petab_select[plot]

2. Open Matlab and setup the Python environment:
   pyenv('Version', path\to\your\Python.exe)

3. To verify the installation, initialize the d2d-PEtabSelect interface in Matlab by calling:
   arPetsInitModule

After running the pyenv command in Matlab the path to your Python installation is remembered. There is no need to set the path again, if you do not change it for some other reason.

If you regularly work with different Python projects, it is recommended to install PEtab Select in a virtual Python environment. In the following we describe how to create a virtual environment with Anaconda Python. Similary, one can also create environments with a standalone Python version (see online resources, e.g. venv or pyenv).

Using a virtual environment with Anaconda Python

If you use the Anaconda Distribution, we advise to create a virtual environment, e.g. with name envPEtabSelect. For this, open up a command window (or Anaconda prompt) and execute the following commands line-by-line:

conda create --name envPEtabSelect Python=3.10
conda activate envPEtabSelect
pip install petab_select[plot]
conda info --envs

If the environment was created successfully, the last command should return a list of all virtual environments, similar to:

# conda environments:
#
base                     C:\ProgramData\anaconda3
...
envPEtabSelect        *  your\custom\path\envPEtabSelect
...

Now open up Matlab and setup the Python environment by calling

pyenv('Version', 'your\custom\path\envPEtabSelect\Python.exe')

Note, that you have to manually append \Python.exe to your\custom\path\envPEtabSelect.

Importing and running PEtab Select problems in d2d

For now we assume that the PEtab Select problem files are already created and the problem.yaml is located at the path PetsProblemFile. By calling the function arPetsSelectModel(PetsProblemFile) the full model selection process is executed automatically: The python module proposes models, which are imported and calibrated in d2d. Then, the calibration results are returned to the python module which proposes new candidate models to calibrate. This loop continues until the new candidate models stop improving the selection criterion and the "best" model ist selected.

The function arPetsSelectModel also takes the name-value argument d2dFitFunction. This is the handle of a function for fitting the models in d2d (e.g. the function handle @arFit). By default a combination of local search and multistart fitting is used.

Note about selection results and multi-start model selection

The model selection algorithms of PEtab Select (except brute force) only explore the model space locally. Therefore, the selected model is a local optimum of the selection criterion. To find the global optimum in model space one should use a multistart approach (similar to multistart fitting with arFitLHS). Automated multi-start model selection is not yet implemented in d2d or PEtab Select. Therefore, one has to manually create multiple copies of the PEtab Select problem and use different model hashes for the predecessor_model in the candidate_space_arguments of the problem.yaml.

Creating PEtab Select problems for your d2d project

In practice we do not only want to solve model selection problems that were predefined in another modeling toolbox. Instead, we want to apply model selection for d2d modeling projects. In the following, we list a few (partly d2d-specific) aspects that should be considered:

• To perform model selection, one has to specify the model selection problem in the selection_problem.yaml and the model space in the model_space.tsv table (see PEtab Select documentation). These files can be created by hand in any text editor, but it might be beneficial to generate them programmatically (see the Examples in: arFramework3/Examples/ToyModels/PEtabSelect/PEtabSelectForD2DModels).

• Before importing the PEtab file (specified in column model_subspace_petab_yaml of the model_space.tsv table) the arPetsSelectModel function tries to load a d2d workspace with matching file name from the Results folder. This behaviour is intended to avoid repeated PEtab import, because one PEtab file is usually shared by multiple candidate models.

• One can also completely avoid the PEtab import by saving the d2d workspaces with arSave(modelName) before starting the model selection. Then, these workspaces can be assigned to model subspaces in the Petab Select model_space.tsv table. For this, the path in the model_subspace_petab_yaml column must point to a valid PEtab file with matching name (i.e. your/custom/path/modelName.yaml). See the examples in: arFramework3/Examples/ToyModels/PEtabSelect/PEtabSelectForD2DModels/

• The column model_subspace_petab_yaml must always be the path of a valid PEtab problem.yaml file. You can create the PEtab files with arExportPEtab. If you avoid the PEtab import (as described above), these PEtab files can also be 'dummy' files (e.g. a simplified version of the model or any other PEtab problem).

• The parameter columns parameter_id must be a subset of the optimization parameters from ar.pLabel. It is not possible to overwrite values that are encoded in the "CONDITIONS" section of the model.def or data.def files.

• Furthermore, the parameters can only be set to fixed numeric values or estimate (if the parameter should be fitted). It is not possible to replace parameters with other parameters or more general expressions. A possible workaround for this limitation can be a clever reparametrization of the d2d model, e.g. by using fold-change parameters and switches. One example for this is the model by Blasi et al. (see arFramework3/Examples/ToyModels/PEtabSelect/TestCases/0009)