Add Admin API (#1137)

* Add Admin API

- Create `admin.py` to serve as Admin API Core
- Create `flask_admin.py` to create flask blueprint for admin API
- Consolidate configuration getter
- Add Pathlib serializing
- Add docker example

* Add integration tests

- Amend admin example to allow writing to configuration. If FS is read only admin API does not work. Returns a 500 and logs `OSError: [Errno 30] Read-only file system: '/pygeoapi/local.config.yml' `

* Preserve env variables in configuration

* Use common accessor functions

- Use common configuration accessor methods for Django and Starlette

* GET returns raw config file

Return configuration with environment variables preserved on GET requests

* Safeguard env variables for root cfg view

"bind": {
"host": "localhost",
"port": "6000"
}
->
"bind": {
"host": "${HOST}",
"port": "${PORT}"
}

* Simplify admin HTML imports

- Use jinja recursion to expand the configuration
- Remove vue from templates

* Create admin API documentation

* Use render_item_value in admin template

* Add Admin API

- Create `admin.py` to serve as Admin API Core
- Create `flask_admin.py` to create flask blueprint for admin API
- Consolidate configuration getter
- Add Pathlib serializing
- Add docker example

* Update GitHub Actions deployment

* Update admin entrypoint

Update admin entrypoint to align with upstream pygeoapi implementation

* Make requested changes

Co-Authored-By: Tom Kralidis <[email protected]>

* Amend test url

Co-Authored-By: Tom Kralidis <[email protected]>

* Fix Admin CI tests

* Add PUT and PATCH for root configuration

- Add put and patch for root configuration
- Add CI tests for PUT and PATCH of root

* Update OpenAPI document wording

* Update entrypoint.sh

Replace tabs with spaces

* Remove unused step

Error from rebasing. Admin API tests are moved to their own job.

* Use jsonpatch

- Use debian supported packaging
- Use custom merge function

* Move test data location

* Create Starlette and Django app

- Fold flask_admin.py into flask_app.py

Co-Authored-By: Tom Kralidis <[email protected]>

* Make requirements-admin.txt

Move admin dependencies to requirements-admin.txt

* Delete guiblock.html

* Update test count for STAC

Update expected test count for addt'l admin test data

* Relegate config warning to config.py

* Move admin tests out of example

* Delete admin docker example

* Update admin-api.rst

* Update pygeoapi-config-0.x.yml

* Update configuration.rst

* Update config.py

* Update admin.py

* Update admin.py

---------

Co-authored-by: Tom Kralidis <[email protected]>

Author: webb-ben

Diff for: .github/workflows/main.yml

@@ -94,6 +94,7 @@ jobs:
     - name: Install requirements 📦
       run: |
         pip3 install -r requirements.txt
+        pip3 install -r requirements-admin.txt
         pip3 install -r requirements-starlette.txt
         pip3 install -r requirements-dev.txt
         pip3 install -r requirements-provider.txt
@@ -151,3 +152,44 @@ jobs:
       if: ${{ failure() }}
       run: |
         pip3 list -v
+
+  admin:
+    needs: [flake8_py3]
+    runs-on: ubuntu-20.04
+    strategy:
+      matrix:
+        include:
+          - python-version: 3.8
+    env:
+      PYGEOAPI_CONFIG: "tests/pygeoapi-test-config-admin.yml"
+      PYGEOAPI_OPENAPI: "tests/pygeoapi-test-openapi-admin.yml"
+    steps:
+    - uses: actions/checkout@v2
+    - uses: actions/setup-python@v2
+      name: Setup Python ${{ matrix.python-version }}
+      with:
+        python-version: ${{ matrix.python-version }}
+    - uses: awalsh128/cache-apt-pkgs-action@latest
+      with:
+        packages: gunicorn python3-gevent
+        version: 1.0
+    - name: Install requirements 📦
+      run: |
+        pip3 install -r requirements.txt
+        pip3 install -r requirements-dev.txt
+        pip3 install -r requirements-admin.txt
+        python3 setup.py install
+    - name: Run pygeoapi with admin API ⚙️
+      run: |
+        pygeoapi openapi generate ${PYGEOAPI_CONFIG} --output-file ${PYGEOAPI_OPENAPI}
+        gunicorn --bind 0.0.0.0:5000 \
+                 --reload \
+                 --reload-extra-file ${PYGEOAPI_CONFIG} \
+                 pygeoapi.flask_app:APP &
+    - name: run integration tests ⚙️
+      run: |
+        pytest tests/test_admin_api.py
+    - name: failed tests 🚩
+      if: ${{ failure() }}
+      run: |
+        pip3 list -v

Diff for: Dockerfile

@@ -64,6 +64,7 @@ ARG ADD_DEB_PACKAGES="\
     python3-elasticsearch \
     python3-fiona \
     python3-gdal \
+    python3-jsonpatch \
     python3-netcdf4 \
     python3-pandas \
     python3-psycopg2 \
@@ -121,6 +122,7 @@ RUN \
 
     # Install remaining pygeoapi deps
     && pip3 install -r requirements-docker.txt \
+    && pip3 install -r requirements-admin.txt \
 
     # Install pygeoapi
     && pip3 install -e . \

Diff for: docs/source/admin-api.rst

@@ -0,0 +1,30 @@
+.. _admin-api:
+
+Admin API
+=========
+
+pygeoapi provides the ability to manage configuration through an API.
+
+When enabled, :ref:`transactions` can be made on pygeoapi's configured resources.  This allows for API based modification of the pygeoapi configuration.
+
+The API is enabled with the following server configuration:
+
+.. code-block:: yaml
+
+    server:
+        admin: true # boolean on whether to enable Admin API.
+
+For pygeoapi to hot reload the configuration as changes are made, the pygeoapi configuration file must be included as
+demonstrated for a gunicorn deployment of pygeoapi via flask:
+
+.. code-block:: bash
+
+    gunicorn \
+        --workers ${WSGI_WORKERS} \
+        --worker-class=${WSGI_WORKER_CLASS} \
+        --timeout ${WSGI_WORKER_TIMEOUT} \
+        --name=${CONTAINER_NAME} \
+        --bind ${CONTAINER_HOST}:${CONTAINER_PORT} \
+        --reload \
+        --reload-extra-file ${PYGEOAPI_CONFIG} \
+        pygeoapi.flask_app:APP

Diff for: docs/source/configuration.rst

@@ -50,6 +50,7 @@ For more information related to API design rules (the ``api_rules`` property in
     cors: true  # boolean on whether server should support CORS
     pretty_print: true  # whether JSON responses should be pretty-printed
     limit: 10  # server limit on number of items to return
+    admin: false  # whether to enable the Admin API
 
     templates: # optional configuration to specify a different set of templates for HTML pages. Recommend using absolute paths. Omit this to use the default provided templates
       path: /path/to/jinja2/templates/folder # path to templates folder containing the Jinja2 template HTML files

Diff for: docs/source/index.rst

@@ -39,6 +39,7 @@ reference documentation on all aspects of the project.
    openapi
    data-publishing/index
    transactions
+   admin-api
    plugins
    html-templating
    crs