notebooks/templates_demo.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Templates demonstration\n",
    "\n",
    "The previous version of numpy used cookiecutter to contruct model execution environment from vanilla cookiecutter setup of a template that was rendered from a cookiecutter.json. This has the advantage of leveraging a lot of cookiecutter features out of the box, but caused some confusion with regards to how configuration was managed. The cookiecutter.json served the purpose as a config file for model configuration, as well as holding the state of the prodecually run wrapper. \n",
    "\n",
    "For this new iteration, we have retained the internal use of cookiecutter in an effort to retain some of the features is provides, but have abstracted it away from the user. All coniguration is now passed an model run instantiation time, all through standard arguments. These arguments then intiate pydantic objects that are used to to things like retrieving and interpolating data inputs if required and then used in a cookiecutter template to render the template. This has been deliberately set up to be general, and flexible, allowing users to either invest in writing high level objects that are quite smart, or to just use the templating engine at a lower level to provide a light python framework around existing configurations. \n",
    "\n",
    "This notebook will highlight some of the aspects of how this has been set up. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Helper functions to dump the contents of input and template\n",
    "from pathlib import Path\n",
    "def dump_input(model):\n",
    "    input_file = Path(model.output_dir) / model.run_id / \"INPUT\"\n",
    "    print(input_file.read_text())\n",
    "\n",
    "def dump_template(model):\n",
    "    template_file = list(Path(model.config.template).glob(\"*\"))[0] / \"INPUT\"\n",
    "    print(template_file.read_text())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:rompy.model:\n",
      "INFO:rompy.model:-----------------------------------------------------\n",
      "INFO:rompy.model:Model settings:\n",
      "INFO:rompy.model:\n",
      "period: \n",
      "\n",
      "\tStart: 2023-01-01 00:00:00\n",
      "\tEnd: 2023-01-04 00:00:00\n",
      "\tDuration: 3 days, 0:00:00\n",
      "\tInterval: 1:00:00\n",
      "\tInclude End: True\n",
      "\n",
      "\n",
      "output_dir: \n",
      "simulations\n",
      "\n",
      "config: \n",
      "grid: \n",
      "\tSwanGrid: REG, 390x150\n",
      "spectral_resolution: \n",
      "\tfmin=0.0464 fmax=1.0 nfreqs=31 ndirs=36\n",
      "forcing: \n",
      "\tbottom: DatasetXarray(uri=simulations/test_swantemplate/datasets/bathy.nc\n",
      "\twind: DatasetXarray(uri=simulations/test_swantemplate/datasets/wind_inputs.nc\n",
      "\tboundary: DatasetXarray(uri=../tests/data/aus-20230101.nc\n",
      "\n",
      "physics: \n",
      "\tfriction='MAD' friction_coeff=0.1\n",
      "outputs: \n",
      "\tGrid:\n",
      "\tvariables: DEPTH UBOT HSIGN HSWELL DIR TPS TM01 WIND\n",
      "\tSpec\n",
      "\t\tlocations: \n",
      "\n",
      "template: \n",
      "\t/source/rompy/rompy/templates/swan\n",
      "\n",
      "INFO:rompy.model:-----------------------------------------------------\n",
      "INFO:rompy.model:Generating model input files in simulations\n",
      "INFO:rompy.swan.config:\t Processing bottom forcing\n",
      "INFO:rompy.swan.data:\tWriting bottom to simulations/test_swantemplate/bottom.grd\n",
      "INFO:rompy.swan.config:\t Processing wind forcing\n",
      "INFO:rompy.swan.data:\tWriting wind to simulations/test_swantemplate/wind.grd\n",
      "INFO:rompy.swan.config:\t Processing boundary forcing\n",
      "INFO:rompy.model:\n",
      "INFO:rompy.model:Successfully generated project in simulations\n",
      "INFO:rompy.model:-----------------------------------------------------\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'/source/rompy/notebooks/simulations/test_swantemplate'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# We will start here with the basic example from the demo.ipynb notebook that you have hopefully already run.\n",
    "# Here we will just initialise the model using the config file and run it \n",
    "from rompy.swan import SwanConfig\n",
    "from rompy import ModelRun\n",
    "import yaml\n",
    "args = yaml.load(open('demo.yml', 'r'), Loader=yaml.FullLoader)\n",
    "run = ModelRun(**args)\n",
    "\n",
    "# and then calling as before\n",
    "run()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'/source/rompy/rompy/templates/swan'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# lets have a look at the contents of the template\n",
    "run.config.template"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "$\n",
      "$ SWAN - Simple example template used by rompy\n",
      "$ Template: {{_template}}\n",
      "$ Generated: {{runtime._generated_at}} on {{runtime._generated_on}} by {{runtime._generated_by}}\n",
      "$ projection: wgs84\n",
      "$\n",
      "\n",
      "MODE NONSTATIONARY TWODIMENSIONAL\n",
      "COORDINATES SPHERICAL\n",
      "SET NAUTICAL\n",
      "\n",
      "{{config.grid}}\n",
      "{{config.forcing['forcing']}}\n",
      "{{config.physics}}\n",
      "{{config.forcing['boundary']}}\n",
      "{{config.outputs}}\n",
      "COMPUTE NONST {{runtime.period.start.strftime(runtime._datefmt)}} {{runtime.frequency}} {{runtime.period.end.strftime(runtime._datefmt)}}\n",
      "\n",
      "STOP\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# There are a few things there, but lets begin with the model control file\n",
    "dump_template(run)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;31mSignature:\u001b[0m \u001b[0mrun\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgenerate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m->\u001b[0m \u001b[0mstr\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mSource:\u001b[0m   \n",
      "    \u001b[0;32mdef\u001b[0m \u001b[0mgenerate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m->\u001b[0m \u001b[0mstr\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0;34m\"\"\"Generate the model input files\u001b[0m\n",
      "\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m        returns\u001b[0m\n",
      "\u001b[0;34m        -------\u001b[0m\n",
      "\u001b[0;34m        staging_dir : str\u001b[0m\n",
      "\u001b[0;34m        \"\"\"\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0mlogger\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minfo\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0mlogger\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minfo\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"-----------------------------------------------------\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0mlogger\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minfo\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Model settings:\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0mlogger\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minfo\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0mlogger\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minfo\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"-----------------------------------------------------\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0mlogger\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minfo\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34mf\"Generating model input files in {self.output_dir}\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0mcc_full\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0mcc_full\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"runtime\"\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdict\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0mcc_full\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"runtime\"\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mupdate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_generation_medatadata\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0mcc_full\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"runtime\"\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mupdate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m{\u001b[0m\u001b[0;34m\"_datefmt\"\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_datefmt\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0;31m# TODO calculate from period\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0mcc_full\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"runtime\"\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"frequency\"\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m\"0.25 HR\"\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0;32mif\u001b[0m \u001b[0mcallable\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconfig\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m            \u001b[0mcc_full\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"config\"\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconfig\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m            \u001b[0mcc_full\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"config\"\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconfig\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0mstaging_dir\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mrender\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m            \u001b[0mcc_full\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconfig\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtemplate\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0moutput_dir\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconfig\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcheckout\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0mlogger\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minfo\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0mlogger\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minfo\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34mf\"Successfully generated project in {self.output_dir}\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0mlogger\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minfo\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"-----------------------------------------------------\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0;32mreturn\u001b[0m \u001b[0mstaging_dir\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mFile:\u001b[0m      /source/rompy/rompy/model.py\n",
      "\u001b[0;31mType:\u001b[0m      method"
     ]
    }
   ],
   "source": [
    "# Of note here is that there are two namespaces, runtime, and config. To explore the differneces, lets look at the generate method of the run\n",
    "run.generate??"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;31mSignature:\u001b[0m \u001b[0mrun\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconfig\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__call__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mruntime\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m->\u001b[0m \u001b[0mstr\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mDocstring:\u001b[0m Call self as a function.\n",
      "\u001b[0;31mSource:\u001b[0m   \n",
      "    \u001b[0;32mdef\u001b[0m \u001b[0m__call__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mruntime\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m->\u001b[0m \u001b[0mstr\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0mret\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0moutputs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgrid\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mperiod\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m            \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0moutputs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgrid\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mperiod\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mruntime\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mperiod\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0moutputs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mspec\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mperiod\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m            \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0moutputs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mspec\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mperiod\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mruntime\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mperiod\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0mret\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"grid\"\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34mf\"{self.domain}\"\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0mret\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"forcing\"\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mforcing\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m            \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgrid\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mruntime\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mperiod\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mruntime\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstaging_dir\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0mret\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"physics\"\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34mf\"{self.physics.cmd}\"\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0mret\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"outputs\"\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0moutputs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcmd\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0mret\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"output_locs\"\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0moutputs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mspec\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlocations\u001b[0m\u001b[0;34m\u001b[0m\n",
      "\u001b[0;34m\u001b[0m        \u001b[0;32mreturn\u001b[0m \u001b[0mret\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mFile:\u001b[0m      /source/rompy/rompy/swan/config.py\n",
      "\u001b[0;31mType:\u001b[0m      method"
     ]
    }
   ],
   "source": [
    "# The cc_full dictionary is the full context used to render the template. The runtime values are populated from the ModelRun object, while the \n",
    "# config values are populated from either the config input. This separation between a model configuration and how it is run means that this config \n",
    "# can be implemneted at varying degrees of complexity. \n",
    "\n",
    "# Lets look at teh current swan implementation as an example. This object is callable, so lets look at the __call__ function to see what it is doing\n",
    "run.config.__call__??"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:rompy.swan.config:\t Processing bottom forcing\n",
      "INFO:rompy.swan.data:\tWriting bottom to simulations/test_swantemplate/bottom.grd\n",
      "INFO:rompy.swan.config:\t Processing wind forcing\n",
      "INFO:rompy.swan.data:\tWriting wind to simulations/test_swantemplate/wind.grd\n",
      "INFO:rompy.swan.config:\t Processing boundary forcing\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'grid': 'CGRID REG 115.68 -32.76 77.0 0.39 0.15 389 149 CIRCLE 36 0.0464 1.0 31\\n\\n',\n",
       " 'forcing': {'forcing': \"INPGRID BOTTOM REG 115.43844490282329 -32.85000000000018 0.0 42 60 0.010000000000005116 0.00999999999999801 EXC -99.0\\nREADINP BOTTOM 1.0 'bottom.grd' 3 FREE\\n\\nINPGRID WIND REG 115.68 -32.76 77.0 389 149 0.001 0.001 NONSTATION 20230101.000000 24.0 HR\\nREADINP WIND 1 'wind.grd' 3 0 1 0 FREE\\n\",\n",
       "  'boundary': \"BOUNDNEST1 NEST 'bnd.bnd' CLOSED\"},\n",
       " 'physics': 'GEN3 WESTH 0.000075 0.00175\\nBREAKING\\nFRICTION MAD 0.1\\n\\nTRIADS\\n\\nPROP BSBT\\nNUM ACCUR 0.02 0.02 0.02 95 NONSTAT 20\\n',\n",
       " 'outputs': \"OUTPUT OPTIONS BLOCK 8\\nBLOCK 'COMPGRID' HEADER 'outputs/swan_out.nc' LAYOUT 1 DEPTH UBOT HSIGN HSWELL DIR TPS TM01 WIND OUT 20230101.000000 1.0 HR\\n\\nPOINTs 'pts' FILE 'out.loc'\\nSPECout 'pts' SPEC2D ABS 'outputs/spec_out.nc' OUTPUT 20230101.000000 1.0 HR\\nTABle 'pts' HEADer 'outputs/tab_out.nc' TIME XP YP HS TPS TM01 DIR DSPR WIND OUTPUT 20230101.000000 1.0 HR\\n\",\n",
       " 'output_locs': OutputLocs}"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# So it is calling the various pydantic components that make up the config which do any work required and return the \n",
    "# relavent values to fill in the template. We have looked at these in a bit more detail in the demo.ipynb notebook, but we can see \n",
    "# populated dictionary by calling the config object directly\n",
    "run.config(runtime=run)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "$\n",
      "$ SWAN - Simple example template used by rompy\n",
      "$ Template: {{_template}}\n",
      "$ Generated: {{runtime._generated_at}} on {{runtime._generated_on}} by {{runtime._generated_by}}\n",
      "$ projection: wgs84\n",
      "$\n",
      "\n",
      "MODE NONSTATIONARY TWODIMENSIONAL\n",
      "COORDINATES SPHERICAL\n",
      "SET NAUTICAL\n",
      "\n",
      "CGRID REG 115.68 -32.76 77.0 0.39 0.15 389 149 CIRCLE 36 0.0464 1.0 31\n",
      "\n",
      "\n",
      "INPGRID BOTTOM REG 115.68 -32.76 0.0 187 150 0.0009999999999999872 0.0009999999999999905 EXC -99.0\n",
      "READINP BOTTOM 1.0 'bottom.grd' 3 FREE\n",
      "\n",
      "INPGRID WIND REG 115.68 -32.76 77.0 389 149 0.001 0.001 NONSTATION {{runtime.period.start.strftime(runtime._datefmt)}} {{runtime.frequency}}\n",
      "READINP WIND 1 'wind.grd' 3 0 1 0 FREE\n",
      "\n",
      "GEN3 WESTH 0.000075 0.00175\n",
      "BREAKING\n",
      "FRICTION MAD {{config.friction_coefficient}}\n",
      "\n",
      "TRIADS\n",
      "\n",
      "PROP BSBT\n",
      "NUM ACCUR 0.02 0.02 0.02 95 NONSTAT 20\n",
      "\n",
      "BOUNDNEST1 NEST 'boundary.bnd' CLOSED\n",
      "OUTPUT OPTIONS BLOCK 8\n",
      "BLOCK 'COMPGRID' HEADER 'outputs/swan_out.nc' LAYOUT 1 DEPTH UBOT HSIGN HSWELL DIR TPS TM01 WIND OUT 20200221.040000 1.0 HR\n",
      "\n",
      "POINTs 'pts' FILE 'out.loc'\n",
      "SPECout 'pts' SPEC2D ABS 'outputs/spec_out.nc' OUTPUT 20200221.040000 1.0 HR\n",
      "TABle 'pts' HEADer 'outputs/tab_out.nc' TIME XP YP HS TPS TM01 DIR DSPR WIND OUTPUT 20200221.040000 1.0 HR\n",
      "\n",
      "COMPUTE NONST {{runtime.period.start.strftime(runtime._datefmt)}} {{runtime.frequency}} {{runtime.period.end.strftime(runtime._datefmt)}}\n",
      "\n",
      "STOP\n"
     ]
    }
   ],
   "source": [
    "# Now this is an example of a medium complexity config object. It is made up of a number of sub componponents grouped into components such as \n",
    "# grid, data, physics etc. Howeever the config and template can be both simpler, and more complex. \n",
    "\n",
    "# As an example. lets consider a simpler case. The repo contains a similar configuration, where much of hte configuration is hard coded. \n",
    "\n",
    "!cat ../rompy/templates/swanbasic/\\{\\{runtime.run_id\\}\\}/INPUT"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:rompy.model:\n",
      "INFO:rompy.model:-----------------------------------------------------\n",
      "INFO:rompy.model:Model settings:\n",
      "INFO:rompy.model:\n",
      "period: \n",
      "\n",
      "\tStart: 2020-02-21 04:00:00\n",
      "\tEnd: 2020-02-24 04:00:00\n",
      "\tDuration: 3 days, 0:00:00\n",
      "\tInterval: 0:15:00\n",
      "\tInclude End: True\n",
      "\n",
      "\n",
      "output_dir: \n",
      "./simulations\n",
      "\n",
      "config: \n",
      "model_type='base' template='../rompy/templates/swanbasic/' checkout='main' friction_coefficient=0.2\n",
      "INFO:rompy.model:-----------------------------------------------------\n",
      "INFO:rompy.model:Generating model input files in ./simulations\n",
      "INFO:rompy.model:\n",
      "INFO:rompy.model:Successfully generated project in ./simulations\n",
      "INFO:rompy.model:-----------------------------------------------------\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'/source/rompy/notebooks/simulations/basic'"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# So in this case, nothing only a single value is rendered from the config object, only runtime object is used. This demonstrates how an existing configuration can be \n",
    "# utilised in Rompy with basic template changes so it could be used to run a hindcast with multiple montthly submissions for example. e.g. to run this \n",
    "# simple case, we just initialise the model with the config pointing to this template\n",
    "\n",
    "basic = ModelRun(run_id='basic', config=dict(template=\"../rompy/templates/swanbasic/\",friction_coefficient=0.2, model_type='base'))\n",
    "\n",
    "# an run it\n",
    "basic()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "$\n",
      "$ SWAN - Simple example template used by rompy\n",
      "$ Template: ../rompy/templates/swanbasic/\n",
      "$ Generated: 2023-06-19 06:15:51.237042 on tom-xps by tdurrant\n",
      "$ projection: wgs84\n",
      "$\n",
      "\n",
      "MODE NONSTATIONARY TWODIMENSIONAL\n",
      "COORDINATES SPHERICAL\n",
      "SET NAUTICAL\n",
      "\n",
      "CGRID REG 115.68 -32.76 77.0 0.39 0.15 389 149 CIRCLE 36 0.0464 1.0 31\n",
      "\n",
      "\n",
      "INPGRID BOTTOM REG 115.68 -32.76 0.0 187 150 0.0009999999999999872 0.0009999999999999905 EXC -99.0\n",
      "READINP BOTTOM 1.0 'bottom.grd' 3 FREE\n",
      "\n",
      "INPGRID WIND REG 115.68 -32.76 77.0 389 149 0.001 0.001 NONSTATION 20200221.040000 0.25 HR\n",
      "READINP WIND 1 'wind.grd' 3 0 1 0 FREE\n",
      "\n",
      "GEN3 WESTH 0.000075 0.00175\n",
      "BREAKING\n",
      "FRICTION MAD 0.2\n",
      "\n",
      "TRIADS\n",
      "\n",
      "PROP BSBT\n",
      "NUM ACCUR 0.02 0.02 0.02 95 NONSTAT 20\n",
      "\n",
      "BOUNDNEST1 NEST 'boundary.bnd' CLOSED\n",
      "OUTPUT OPTIONS BLOCK 8\n",
      "BLOCK 'COMPGRID' HEADER 'outputs/swan_out.nc' LAYOUT 1 DEPTH UBOT HSIGN HSWELL DIR TPS TM01 WIND OUT 20200221.040000 1.0 HR\n",
      "\n",
      "POINTs 'pts' FILE 'out.loc'\n",
      "SPECout 'pts' SPEC2D ABS 'outputs/spec_out.nc' OUTPUT 20200221.040000 1.0 HR\n",
      "TABle 'pts' HEADer 'outputs/tab_out.nc' TIME XP YP HS TPS TM01 DIR DSPR WIND OUTPUT 20200221.040000 1.0 HR\n",
      "\n",
      "COMPUTE NONST 20200221.040000 0.25 HR 20200224.040000\n",
      "\n",
      "STOP\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# If we inspect the control file, we should see the run times have been popoluated by the runtime, and the friction coefficient has been set to 0.2. \n",
    "# No python code was required to do this, just a template and a dictionary or required arguments. \n",
    "dump_input(basic)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Config objects can also go the other way. At the moment, only a subset of the SWAN configuration is implemented in Rompy. However, it is possible to\n",
    "# create pydantic objects similar to those on teh SwanConfig object, but mirroring the entire functionaly of the SWAN model. This is a work in progress,\n",
    "# and will be demonstrated in a this notebook components/swan-config-components.ipynb.\n",
    "\n",
    "# Hopefully this shows that the model config object can be as simple or as complex as requuired. This also provided a soft on ramp to developiong \n",
    "# support for other models as you are not required to write do weeks of development before you can even start. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.10"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}