Merge pull request #72 from ESA-VirES/staging

smithara · web-flow · commit 5906ff915940 · 2022-02-08T21:46:55.000Z
Merge to master branch to prepare next release
diff --git a/.github/workflows/CI.yml b/.github/workflows/CI.yml
@@ -0,0 +1,25 @@
+name: CI
+
+on:
+  push:
+    branches: [ staging ]
+  pull_request:
+    branches: [ staging ]
+  workflow_dispatch:
+
+jobs:
+  run-tests:
+    runs-on: ${{ matrix.operating-system }}
+    strategy:
+      matrix:
+        operating-system:
+          - ubuntu-latest
+          - windows-latest
+          - macOS-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v2
+      - name: Setup Nox
+        uses: excitedleigh/setup-nox@v2.0.0
+      - name: Run Nox
+        run: nox
diff --git a/.gitignore b/.gitignore
@@ -32,3 +32,6 @@ MANIFEST
 
 # Sphinx documentation
 docs/_build/
+
+# nox
+.nox
diff --git a/.travis.yml b/.travis.yml
diff --git a/README.rst b/README.rst
@@ -10,33 +10,31 @@
     :target: https://requires.io/github/ESA-VirES/VirES-Python-Client/requirements/?branch=master
     :alt: Requirements Status
 
-.. image:: https://travis-ci.org/ESA-VirES/VirES-Python-Client.svg?branch=master
-    :target: https://travis-ci.org/ESA-VirES/VirES-Python-Client
-
-.. image:: https://coveralls.io/repos/github/ESA-VirES/VirES-Python-Client/badge.svg
-    :target: https://coveralls.io/github/ESA-VirES/VirES-Python-Client
-
 .. image:: https://zenodo.org/badge/138034133.svg
    :target: https://zenodo.org/badge/latestdoi/138034133
 
 ::
 
   pip install viresclient
 
-viresclient_ is a Python package which connects to a VirES_ server through the WPS_ interface and handles product requests and downloads. This enables easy access to ESA's `Swarm mission`_ data and models. This service is provided for ESA by EOX_. For enquiries about the service and problems with accessing your account, please email info@vires.services. For help with usage, please email ashley.smith@ed.ac.uk or `raise an issue on GitHub`_.
+viresclient_ is a Python package which connects to a VirES server, of which there are two: *VirES for Swarm* (https://vires.services) and *VirES for Aeolus* (https://aeolus.services), through the WPS_ interface. This package handles product requests and downloads, enabling easy access to data and models from ESA's Earth Explorer missions, Swarm_ and Aeolus_. This service is provided for ESA by EOX_. For enquiries about the service and problems with accessing your account, please email info@vires.services. For help with usage, please email ashley.smith@ed.ac.uk (for Swarm data) or `raise an issue on GitHub`_.
 
 .. _viresclient: https://github.com/ESA-VirES/VirES-Python-Client
-.. _VirES: https://vires.services
 .. _WPS: http://www.opengeospatial.org/standards/wps
-.. _`Swarm mission`: https://earth.esa.int/eogateway/missions/swarm
+.. _Swarm: https://earth.esa.int/eogateway/missions/swarm
+.. _Aeolus: https://earth.esa.int/eogateway/missions/aeolus
 .. _EOX: https://eox.at/category/vires/
 .. _`raise an issue on GitHub`: https://github.com/ESA-VirES/VirES-Python-Client/issues
 
-Data and models are processed on demand on the server - a combination of measurements from any time interval can be accessed. These are the same data that can be accessed by the `VirES GUI`_. *viresclient* handles the returned data to allow direct loading as a single pandas.DataFrame_, or xarray.Dataset_.
+For code recipes and more, see `Swarm Notebooks`_ & `Aeolus Notebooks`_. To start experimenting right away, *viresclient* is installed on the "Virtual Research Environment" (VRE), which is a managed Jupyter-based system provided for ESA by EOX. The service is free and open to all, accessible through your VirES account - check the notebooks to read more and get started.
+
+.. _`Swarm Notebooks`: https://swarm.magneticearth.org
+.. _`Aeolus Notebooks`: https://notebooks.aeolus.services
+
+Data and models are processed on demand on the VirES server - a combination of measurements from any time interval can be accessed. These are the same data that can be accessed by the VirES GUI. *viresclient* handles the returned data to allow direct loading as a single pandas.DataFrame_, or xarray.Dataset_.
 
 .. _pandas.DataFrame: https://pandas.pydata.org/pandas-docs/stable/dsintro.html#dataframe
 .. _xarray.Dataset: http://xarray.pydata.org/en/stable/data-structures.html#dataset
-.. _`VirES GUI`: https://vires.services
 
 .. code-block:: python
 
@@ -91,10 +89,7 @@ Data and models are processed on demand on the server - a combination of measure
    RangeFilters:    []
 
 
-*viresclient* is installed on the `"Virtual Research Environment" (VRE)`_, which is a managed Jupyter-based system provided for ESA by EOX. The service is free and open to all. See `"Swarm Notebooks"`_ for how-to guides with Swarm data and guidance in using the VRE.
 
-.. _`"Virtual Research Environment" (VRE)`: https://vre.vires.services
-.. _`"Swarm Notebooks"`: https://swarm.magneticearth.org
 
 .. image:: https://github.com/ESA-VirES/Swarm-VRE/raw/master/docs/images/VRE_shortest_demo.gif
 
@@ -105,3 +100,7 @@ How to acknowledge VirES
 You can reference *viresclient* directly using the DOI of our zenodo_ record. VirES uses data from a number of different sources so please also acknowledge these appropriately.
 
 .. _zenodo: https://doi.org/10.5281/zenodo.2554162
+
+    | "We use the Python package, viresclient [1], to access [...] from ESA's VirES for Swarm service [2]"  
+    | [1] https://doi.org/10.5281/zenodo.2554162  
+    | [2] https://vires.services
diff --git a/docs/access_token.rst b/docs/access_token.rst
@@ -5,7 +5,7 @@ An access token is required to access the VirES server.
 The tokens are managed via the web user interface.
 
 Assuming you have an existing VirES account and you can access
-the VirES web client (`htts://vires.services <https://vires.services>`_),
+the VirES web client (https://vires.services or https://aeolus.services),
 you get to token manager via `Manage access tokens` item in your account menu
 (displaying your username)
 
diff --git a/docs/api.rst b/docs/api.rst
@@ -8,6 +8,15 @@ SwarmRequest
     :inherited-members:
     :exclude-members: AUXILIARY_VARIABLES, COLLECTIONS, COLLECTION_SAMPLING_STEPS, MAGNETIC_MODELS, MAGNETIC_MODEL_VARIABLES, OBS_COLLECTIONS, PRODUCT_VARIABLES
 
+AeolusRequest
+=============
+
+.. autoclass:: viresclient.AeolusRequest
+    :members:
+    :undoc-members:
+    :show-inheritance:
+    :inherited-members:
+
 ReturnedData
 ============
 
diff --git a/docs/available_parameters.rst b/docs/available_parameters.rst
@@ -1,5 +1,5 @@
-Available parameters for Swarm data
-===================================
+Available parameters for Swarm
+==============================
 
 .. note::
 
diff --git a/docs/available_parameters_aeolus.rst b/docs/available_parameters_aeolus.rst
@@ -0,0 +1,5 @@
+Available parameters for Aeolus
+===============================
+
+.. note::
+  Details to follow. See https://notebooks.aeolus.services for examples.
diff --git a/docs/config_details.rst b/docs/config_details.rst
@@ -1,8 +1,28 @@
 Configuration Details
 =====================
 
-.. note::
-  Be careful not to accidentally add your credentials to online repositories or containers. You can use the CLI command ``viresclient clear_credentials`` to remove them.
+.. attention::
+  Be careful not to accidentally add your credentials to online repositories or containers. You can use the CLI command ``viresclient clear_credentials`` to remove them from your environment. By default, this is just a file located at ``~/.viresclient.ini``, where ``~`` is your home directory which is dependent on the operating system.
+
+.. tip::
+
+  To get started quickly in Jupyter notebooks:
+
+  .. code-block:: python
+
+    from viresclient import SwarmRequest
+    r = SwarmRequest("https://vires.services/ows")
+
+  OR:
+
+  .. code-block:: python
+
+    from viresclient import AeolusRequest
+    r = AeolusRequest("https://aeolus.services/ows")
+
+  ... then follow automatic instructions to configure token if not already set
+
+NB: URL's on this page assume using *VirES for Swarm*. If using *Aeolus* instead, replace ``https://vires.services/ows`` with ``https://aeolus.services/ows``.
 
 While it is possible to enter the server URL and access credentials (see :doc:`access_token`) each time a new request object is created,
 
diff --git a/docs/docs_requirements.txt b/docs/docs_requirements.txt
@@ -6,3 +6,4 @@ cdflib
 tables
 tqdm
 xarray
+netcdf4
diff --git a/docs/index.rst b/docs/index.rst
@@ -4,11 +4,13 @@ Welcome to viresclient's documentation!
 Get started `at the introduction <readme.html>`_
 
 .. toctree::
-  :caption: Virtual Research Environment
+  :caption: Virtual Research Environments
 
-  🔗 [Launch VRE / JupyterLab] <https://vre.vires.services>
+  🔗 VirES for Swarm <https://vires.services>
   🔗 How-To Guides [Swarm Notebooks] <https://swarm.magneticearth.org>
-  🔗 VirES Graphical Interface <https://vires.services>
+  🔗 VirES for Aeolus <https://aeolus.services>
+  🔗 How-To Guides [Aeolus Notebooks] <https://notebooks.aeolus.services>
+  
 
 .. toctree::
   :maxdepth: 2
@@ -19,6 +21,7 @@ Get started `at the introduction <readme.html>`_
   config_details
   access_token
   available_parameters
+  available_parameters_aeolus
   notebook_intro
   release_notes
 
diff --git a/docs/installation.rst b/docs/installation.rst
@@ -1,13 +1,11 @@
 Installation and First Usage
 ============================
 
+.. note:: For VRE users (it's free! read more: `Swarm <https://swarm.magneticearth.org>`_, `Aeolus <https://notebooks.aeolus.services>`_), viresclient is already installed and configured so skip these steps
+
 1. Installation
 ---------------
 
-.. note:: For VRE (Virtual Research Environment) users (it's free! - `read more <https://swarm.magneticearth.org>`_):
-
-  viresclient is already installed so skip this step. Log in at https://vre.vires.services and refer to `the introduction notebooks <https://swarm.magneticearth.org/notebooks/02a__intro-swarm-viresclient>`_.
-
 Python ≥ 3.6 is required. Testing is primarily on Linux, but macOS and Windows should also work.
 
 It can currently be installed with::
@@ -23,10 +21,11 @@ Dependencies::
   cdflib
   pandas
   xarray
+  netCDF4
 
 pip will fetch these automatically - if you are using conda, it may be better to install these first using conda instead (where available)::
 
-    conda install requests jinja2 pytables tqdm pandas xarray
+    conda install requests jinja2 pytables tqdm pandas xarray netcdf4
     pip install viresclient
 
 Recommended setup if starting without Python already
@@ -35,7 +34,7 @@ Recommended setup if starting without Python already
 1. Install Miniconda: https://docs.conda.io/en/latest/miniconda.html
 2. Create a new conda environment with some recommended packages::
 
-    conda create --name myenv scipy matplotlib pandas xarray cartopy jupyter jupyterlab flake8 dask h5py netCDF4 jinja2 pytables tqdm
+    conda create --name myenv scipy matplotlib pandas xarray cartopy jupyter jupyterlab flake8 dask h5py netCDF4 jinja2 pytables tqdm ipywidgets
 
   Activate the new environment (you do this each time you want to use it)::
 
diff --git a/docs/notebook_intro.rst b/docs/notebook_intro.rst
@@ -1,48 +1,32 @@
 Introduction to notebooks
 =========================
 
-Jupyter notebooks are a convenient tool for interactive data exploration, rapid prototyping, and producing reports. The Virtual Research Environment provides free JupyterLab instances with persistent storage where you can run notebooks working with Swarm data.
-
+Jupyter notebooks are a convenient tool for interactive data exploration, rapid prototyping, and producing reports. The Virtual Research Environment (VRE) provides free JupyterLab instances with persistent storage where you can run notebooks.
 
+The VRE is a whole analysis suite, including many other libraries from the Python ecosystem. We provide recipes in the form of noteboooks compiled into a "Jupyter Book" to show how viresclient can be blended together with other libraries to access, visualise, and analyse data.
 
 .. list-table:: Notebook repositories
    :header-rows: 1
 
-   *  -  Name (GitHub Link)
-      -  View (static/docs)
-      -  Launch/interact
-      -  Description
+   *  -  Name / GitHub
+      -  View
+      -  Interact
    *  -  `Swarm-DISC/Swarm_notebooks <https://github.com/Swarm-DISC/Swarm_notebooks>`_
-      -  .. image:: https://img.shields.io/badge/render-JupyterBook-orange.svg
+      -  .. image:: https://jupyterbook.org/badge.svg
             :target: https://swarm.magneticearth.org
-      -  .. image:: https://img.shields.io/badge/nbgitpuller-VRE-blue
+      -  .. image:: https://img.shields.io/badge/nbgitpuller-VRE--Swarm-blue
             :target: https://vre.vires.services/hub/user-redirect/git-pull?repo=https://github.com/Swarm-DISC/Swarm_notebooks&urlpath=lab/tree/Swarm_notebooks/notebooks/
-      -  Curated Swarm notebooks for scientists
-   *  -  `pacesm/jupyter_notebooks <https://github.com/pacesm/jupyter_notebooks>`_
-      -  .. image:: https://img.shields.io/badge/render-nbviewer-orange.svg
-            :target: https://nbviewer.jupyter.org/github/pacesm/jupyter_notebooks
-      -  .. image:: https://img.shields.io/badge/nbgitpuller-VRE-blue
-            :target: https://vre.vires.services/hub/user-redirect/git-pull?repo=https://github.com/pacesm/jupyter_notebooks&urlpath=lab/tree/jupyter_notebooks/
-      -  Reference mainly during new product registration and configuration
-
+   *  -  `ESA-VirES/Aeolus-notebooks <https://github.com/ESA-VirES/Aeolus-notebooks>`_
+      -  .. image:: https://jupyterbook.org/badge.svg
+            :target: https://notebooks.aeolus.services
+      -  .. image:: https://img.shields.io/badge/nbgitpuller-VRE--Aeolus-blue
+            :target: https://vre.aeolus.services/hub/user-redirect/git-pull?repo=https://github.com/ESA-VirES/Aeolus-notebooks&urlpath=lab/tree/Aeolus-notebooks/notebooks&branch=main
 
 .. note::
 
-  Sometimes notebooks won't render directly on the GitHub website (or are slow). Try `nbviewer <https://nbviewer.jupyter.org/>`_ instead (see the "Render" links above).
-  
-  In the case of ``Swarm_notebooks``, the notebooks are stored in the repository *without outputs included*, so are better viewed at `swarm.magneticearth.org <https://swarm.magneticearth.org>`_ (the *Jupyter Book* link above)
-
-Notebooks can be uploaded to JupyterLab using the "Upload" button (which means you must first download the notebooks to your computer from GitHub). To easily access a full repository, open a Terminal and use git:
-
-To clone a repository to your working space::
-
-    git clone https://github.com/Swarm-DISC/Swarm_notebooks.git
-
-(this will clone it into ``Swarm_notebooks`` within your current directory)
-
-To clear any changes you made and fetch the latest version, from within ``Swarm_notebooks`` run::
+      The *nbgitpuller* links above perform *git* operations for you, applying updates when you re-click the link using special `automatic merging behaviour <https://jupyterhub.github.io/nbgitpuller/topic/automatic-merging.html>`_. Sometimes it may be necessary to perform the git operations directly instead.
 
-    git fetch
-    git reset --hard origin/master
+      To clear any changes you made and fetch the latest version, from within ``Swarm_notebooks`` or ``Aeolus-notebooks`` run from a terminal::
 
-The *nbgitpuller* links above perform a ``git clone`` operation for you, and applies updates when you re-click the link using special `automatic merging behaviour <https://jupyterhub.github.io/nbgitpuller/topic/automatic-merging.html>`_. Sometimes it may be necessary to perform the git operations directly instead.
+            git fetch
+            git reset --hard origin/master
diff --git a/docs/release_notes.rst b/docs/release_notes.rst
@@ -4,6 +4,12 @@ Release notes
 Change log
 ----------
 
+Changes from 0.9.1 to 0.10.0
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+- Added functionality to support VirES for Aeolus. See https://notebooks.aeolus.services
+- Added dependency: `netCDF4 <https://github.com/Unidata/netcdf4-python>`_
+
 Changes from 0.9.0 to 0.9.1
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
diff --git a/noxfile.py b/noxfile.py
@@ -0,0 +1,7 @@
+import nox
+
+@nox.session(python=["3.6", "3.7", "3.8", "3.9", "3.10"])
+def test(session):
+    session.install(".")
+    session.install("pytest")
+    session.run("pytest")
diff --git a/setup.py b/setup.py
@@ -71,6 +71,7 @@ def read(fname):
         'tqdm>=4.23.0, <5.0.0',
         'xarray>=0.11.0, <1.0.0',
         'requests>=2.0.0, <3.0.0',
+        'netCDF4>=1.5.3, <1.6.0',
     ],
     entry_points={
         'console_scripts': [
diff --git a/viresclient/__init__.py b/viresclient/__init__.py
@@ -37,4 +37,4 @@
 from . import _data
 
 
-__version__ = "0.9.1"
+__version__ = "0.10.0"
diff --git a/viresclient/_client.py b/viresclient/_client.py
@@ -93,6 +93,11 @@
 For more details, see:
 https://viresclient.readthedocs.io/en/latest/config_details.html"""
 
+PRODUCTION_URLS = {
+    "Swarm": "https://vires.services/ows",
+    "Aeolus": "https://aeolus.services/ows",
+}
+
 
 
 def get_log_level(level):
@@ -250,18 +255,20 @@ class ClientRequest(object):
     def __init__(self, url=None, token=None,
                  config=None, logging_level="NO_LOGGING", server_type=None):
 
+        self._server_type = server_type
+
         # Check and prompt for token if not already set, then store in config
         # Try to only do this if running in a notebook
         if IN_JUPYTER:
             if not (token or config):
                 cc = ClientConfig()
                 # Use production url if none chosen
-                url = url or cc.default_url or "https://vires.services/ows"
+                prod_url = PRODUCTION_URLS.get(self._server_type, None)
+                url = url or cc.default_url or prod_url
                 if not cc.get_site_config(url):
                     print("Access token not found.")
                     set_token(url)
 
-        self._server_type = server_type
         self._available = {}
         self._collection = None
         self._request_inputs = None
diff --git a/viresclient/_client_aeolus.py b/viresclient/_client_aeolus.py
diff --git a/viresclient/_data/__init__.py b/viresclient/_data/__init__.py
diff --git a/viresclient/_data/config_aeolus.json b/viresclient/_data/config_aeolus.json
diff --git a/viresclient/_data_handling.py b/viresclient/_data_handling.py
diff --git a/viresclient/_wps/templates/vires_aeolus_fetch_filtered_data.xml b/viresclient/_wps/templates/vires_aeolus_fetch_filtered_data.xml
diff --git a/viresclient/_wps/templates/vires_aeolus_fetch_filtered_data_async.xml b/viresclient/_wps/templates/vires_aeolus_fetch_filtered_data_async.xml

-Original file line number
+Diff line change
 tables
 tqdm
 xarray
 +netcdf4