fix CanarieAPI config to support Python 3.x (#284)

## Overview

Please include a summary of the changes and which issues are fixed. 

Please also include relevant motivation and context. 

List any dependencies that are required for this change.

## Breaking changes

- CanarieAPI: update to `0.7.1`.

  - The Docker running `CanarieAPI` is now using Python 3 (since `0.4.x` tags).
    Configurations need to be updated if any specific Python 2 definitions were used.
    See [2to3](https://docs.python.org/3/library/2to3.html) to help migrate configurations automatically if necessary.
  - Update the [CanarieAPI configuration](birdhouse/config/canarie-api/docker_configuration.py.template) to use
    Python 3.x executable code.

## Changes

- CanarieAPI: update to `0.7.1`.

  - The server node now provides a generic ``server`` configuration for the current ``platform`` definition.
  - Added multiple missing docuementation references for all the services included within `CanarieAPI` configurations.
  - With new `CanarieAPI` version, a slightly improved UI with more service details are provided for the active server:

![image](https://user-images.githubusercontent.com/19194484/232822454-e39c0111-54dc-4f9b-adf6-5ea6e59d67e3.png)

- Add optional variables witht defaults to define reference Docker image version tags.

  Following optional variables are defined by default. These are used as reference in the respective Docker compose
  service definition of these components, as well as in their `CanarieAPI` configuration to retrieve the release time
  of the tag, and refer to relevant URL references as needed.

  - `CATALOG_VERSION`
  - `FINCH_VERSION`
  - `FLYINGPIGEON_VERSION`
  - `GEOSERVER_VERSION`
  - `HUMMINGBIRD_VERSION`
  - `MALLEEFOWL_VERSION`
  - `RAVEN_VERSION`

## Fixes:

- CanarieAPI: update to `0.7.1`.

  - Fixes an `AttributeError` raised due to misconfiguration of the Web Application with Flask 2.x definitions
    (relates to [Ouranosinc/CanarieAPI#10](Ouranosinc/CanarieAPI#10)).
  - Skip over `0.4.x`, `0.5.x`, `0.6.x`  versions to avoid issue related to `cron` job monitoring and log parser
    command failures in order to collect configured service statistics and statuses
    (see also [Ouranosinc/CanarieAPI#14](Ouranosinc/CanarieAPI#14)).

- Weaver: update CanarieAPI monitoring definitions
  - Move monitoring of public endpoint under [optional-components/canarie-api-full-monitoring][canarie-monitor].
  - Add monitoring of private endpoint by default when using Weaver component.

- Cowbird: update CanarieAPI monitoring definitions
  - Add monitoring of public endpoint under [optional-components/canarie-api-full-monitoring][canarie-monitor].
  - Add public Magpie permission on Cowbird entrypoint only to allow its monitoring.

## Additional Information

Resolves the following log error.

```
proxy              | [2023-01-31 19:37:01 +0000] [37] [DEBUG] GET /canarie/
proxy              | [2023-01-31 19:37:01,708] [37] [INFO] app_object : Disconnecting from database
proxy              | [2023-01-31 19:37:01,709] [37] [DEBUG] app_object : Using db filename : /opt/local/src/CanarieAPI/stats.db
proxy              | [2023-01-31 19:37:01 +0000] [37] [DEBUG] Closing connection. 
proxy              | [2023-01-31 19:37:02 +0000] [37] [DEBUG] GET /canarie/background.jpg
proxy              | [2023-01-31 19:37:02,176] [37] [INFO] app_object : Disconnecting from database
proxy              | [2023-01-31 19:37:02,176] [37] [DEBUG] app_object : Using db filename : /opt/local/src/CanarieAPI/stats.db
proxy              | [2023-01-31 19:37:02 +0000] [37] [ERROR] Error handling request /canarie/background.jpg
proxy              | Traceback (most recent call last):
proxy              |   File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/async.py", line 56, in handle
proxy              |     self.handle_request(listener_name, req, client, addr)
proxy              |   File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/ggevent.py", line 152, in handle_request
proxy              |     super(GeventWorker, self).handle_request(*args)
proxy              |   File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/async.py", line 107, in handle_request
proxy              |     respiter = self.wsgi(environ, resp.start_response)
proxy              |   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1997, in __call__
proxy              |     return self.wsgi_app(environ, start_response)
proxy              |   File "/opt/local/src/CanarieAPI/canarieapi/reverse_proxied.py", line 33, in __call__
proxy              |     return self.app(environ, start_response)
proxy              |   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1985, in wsgi_app
proxy              |     response = self.handle_exception(e)
proxy              |   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1532, in handle_exception
proxy              |     handler = self._find_error_handler(InternalServerError())
proxy              |   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1449, in _find_error_handler
proxy              |     .get(code))
proxy              |   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1440, in find_handler
proxy              |     handler = handler_map.get(cls)
proxy              | AttributeError: 'function' object has no attribute 'get'
```

The above problem lead to unresponsive CanarieAPI, which in turn caused the platform to fail responding with successful monitoring statuses for requests toward the configured services, which in turn, caused the stack to never completely boot. When the stack failed to boot, the *End2End Test Results* (example: #283 (comment)) could not run due to the unresponsive instance, which is the cause of the incomplete output:

`````
Tests URL :

NOTEBOOK TEST RESULTS
   
````
</code>
``
`````

# To Do (in follow-up PRs)

- remove deprecated configs deleted in #287 
- remove deprecated configs deleted in #291 
- remove deprecated configs deleted in #292

Author: fmigneault

.bumpversion.cfg

@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 1.25.7
+current_version = 1.26.0
 commit = True
 tag = False
 tag_name = {new_version}
@@ -30,11 +30,11 @@ search = {current_version}
 replace = {new_version}
 
 [bumpversion:file:RELEASE.txt]
-search = {current_version} 2023-04-20T14:15:34Z
+search = {current_version} 2023-04-20T20:42:31Z
 replace = {new_version} {utcnow:%Y-%m-%dT%H:%M:%SZ}
 
 [bumpversion:part:releaseTime]
-values = 2023-04-20T14:15:34Z
+values = 2023-04-20T20:42:31Z
 
 [bumpversion:file(version):birdhouse/config/canarie-api/docker_configuration.py.template]
 search = 'version': '{current_version}'

CHANGES.md

@@ -17,6 +17,64 @@
 
 [//]: # (list changes here, using '-' for each new entry, remove this when items are added)
 
+[1.26.0](https://github.com/bird-house/birdhouse-deploy/tree/1.26.0) (2023-04-20)
+------------------------------------------------------------------------------------------------------------------
+
+
+## Breaking changes
+
+- CanarieAPI: update to `0.7.1`.
+
+  - The Docker running `CanarieAPI` is now using Python 3 (since `0.4.x` tags).
+    Configurations need to be updated if any specific Python 2 definitions were used.
+    See [2to3](https://docs.python.org/3/library/2to3.html) to help migrate configurations automatically if necessary.
+  - Update the [CanarieAPI configuration](birdhouse/config/canarie-api/docker_configuration.py.template) to use
+    Python 3.x executable code.
+
+## Changes
+
+- CanarieAPI: update to `0.7.1`.
+
+  - The server node now provides a generic ``server`` configuration for the current ``platform`` definition.
+  - Added multiple missing docuementation references for all the services included within `CanarieAPI` configurations.
+  - With new `CanarieAPI` version, a slightly improved UI with more service details are provided for the active server:
+
+![image](https://user-images.githubusercontent.com/19194484/232822454-e39c0111-54dc-4f9b-adf6-5ea6e59d67e3.png)
+
+- Add optional variables witht defaults to define reference Docker image version tags.
+
+  Following optional variables are defined by default. These are used as reference in the respective Docker compose
+  service definition of these components, as well as in their `CanarieAPI` configuration to retrieve the release time
+  of the tag, and refer to relevant URL references as needed.
+
+  - `CATALOG_VERSION`
+  - `FINCH_VERSION`
+  - `FLYINGPIGEON_VERSION`
+  - `GEOSERVER_VERSION`
+  - `HUMMINGBIRD_VERSION`
+  - `MALLEEFOWL_VERSION`
+  - `RAVEN_VERSION`
+
+## Fixes:
+
+- CanarieAPI: update to `0.7.1`.
+
+  - Fixes an `AttributeError` raised due to misconfiguration of the Web Application with Flask 2.x definitions
+    (relates to [Ouranosinc/CanarieAPI#10](https://github.com/Ouranosinc/CanarieAPI/pull/10)).
+  - Skip over `0.4.x`, `0.5.x`, `0.6.x`  versions to avoid issue related to `cron` job monitoring and log parser
+    command failures in order to collect configured service statistics and statuses
+    (see also [Ouranosinc/CanarieAPI#14](https://github.com/Ouranosinc/CanarieAPI/pull/14)).
+
+- Weaver: update CanarieAPI monitoring definitions
+  - Move monitoring of public endpoint under [optional-components/canarie-api-full-monitoring][canarie-monitor].
+  - Add monitoring of private endpoint by default when using Weaver component.
+
+- Cowbird: update CanarieAPI monitoring definitions
+  - Add monitoring of public endpoint under [optional-components/canarie-api-full-monitoring][canarie-monitor].
+  - Add public Magpie permission on Cowbird entrypoint only to allow its monitoring.
+
+[canarie-monitor]: birdhouse/optional-components/canarie-api-full-monitoring
+
 [1.25.7](https://github.com/bird-house/birdhouse-deploy/tree/1.25.7) (2023-04-20)
 ------------------------------------------------------------------------------------------------------------------
 

Makefile

@@ -1,7 +1,7 @@
 # Generic variables
 override SHELL       := bash
 override APP_NAME    := birdhouse-deploy
-override APP_VERSION := 1.25.7
+override APP_VERSION := 1.26.0
 
 # utility to remove comments after value of an option variable
 override clean_opt = $(shell echo "$(1)" | $(_SED) -r -e "s/[ '$'\t'']+$$//g")

README.rst

@@ -14,13 +14,13 @@ for a full-fledged production platform.
     * - releases
       - | |latest-version| |commits-since|
 
-.. |commits-since| image:: https://img.shields.io/github/commits-since/bird-house/birdhouse-deploy/1.25.7.svg
+.. |commits-since| image:: https://img.shields.io/github/commits-since/bird-house/birdhouse-deploy/1.26.0.svg
     :alt: Commits since latest release
-    :target: https://github.com/bird-house/birdhouse-deploy/compare/1.25.7...master
+    :target: https://github.com/bird-house/birdhouse-deploy/compare/1.26.0...master
 
-.. |latest-version| image:: https://img.shields.io/badge/tag-1.25.7-blue.svg?style=flat
+.. |latest-version| image:: https://img.shields.io/badge/tag-1.26.0-blue.svg?style=flat
     :alt: Latest Tag
-    :target: https://github.com/bird-house/birdhouse-deploy/tree/1.25.7
+    :target: https://github.com/bird-house/birdhouse-deploy/tree/1.26.0
 
 .. |readthedocs| image:: https://readthedocs.org/projects/birdhouse-deploy/badge/?version=latest
     :alt: ReadTheDocs Build Status (latest version)

RELEASE.txt

@@ -1 +1 @@
-1.25.7 2023-04-20T14:15:34Z
+1.26.0 2023-04-20T20:42:31Z

birdhouse/components/cowbird/.gitignore

@@ -3,6 +3,8 @@ config/cowbird/config.yml
 config/cowbird/cowbird.ini
 config/magpie/config.yml
 config/proxy/conf.extra-service.d/cowbird.conf
+config/proxy/canarie_api_monitoring.py
 
 # Old paths. Keep these so that old config files remain uncommittable after updates.
 conf.extra-service.d/cowbird.conf
+config/canarie-api/cowbird_config.py

birdhouse/components/cowbird/config/magpie/config.yml.template

@@ -18,6 +18,17 @@ permissions:
     permission: write
     group: administrators
     action: create
+  # Allow validating/monitoring the service entrypoint and its version
+  # Everything else should remain admin-protected since it manages other user/group permissions.
+  - service: cowbird
+    permission: read-match
+    group: anonymous
+    action: create
+  - service: cowbird
+    resource: /version
+    permission: read-match
+    group: anonymous
+    action: create
 
 # Definitions of all the webhooks urls that will be called when creating or deleting a user.
 webhooks:

birdhouse/components/cowbird/config/proxy/canarie_api_monitoring.py.template

@@ -0,0 +1,38 @@
+SERVICES['Cowbird'] = {
+    'info': {
+        'name': 'Cowbird',
+        'synopsis': 'Cowbird is a middleware that manages interactions between various birds of the bird-house stack.',
+        'version': "${COWBIRD_VERSION}",
+        'institution': 'Ouranos, CRIM',
+        'releaseTime': get_release_time_from_repo_tag("github", "Ouranosinc/cowbird", "${COWBIRD_VERSION}"),
+        'researchSubject': 'Any',
+        'supportEmail': '${SUPPORT_EMAIL}',
+        'category': 'Security',
+        'tags': ['Security', 'Management', 'Access', 'Policy Decision Point']
+    },
+    'stats': {
+        'method': '.*',
+        'route': "/cowbird/.*"
+    },
+    'redirect': {
+        'doc': 'https://pavics-cowbird.readthedocs.io/',
+        'releasenotes': 'https://github.com/Ouranosinc/cowbird//blob/master/CHANGES.rst',
+        'support': 'https://github.com/Ouranosinc/cowbird//issues',
+        'source': 'https://github.com/Ouranosinc/cowbird/',
+        'tryme': 'https://${PAVICS_FQDN_PUBLIC}/cowbird/',
+        'licence': 'https://github.com/Ouranosinc/cowbird//blob/${COWBIRD_VERSION}/LICENSE',
+        'provenance': 'https://github.com/Ouranosinc/cowbird/'
+    },
+    "monitoring": {
+        "Cowbird": {
+            'request': {
+                'url': 'http://cowbird:7000/'
+            },
+            'response': {
+                'text': '\{.*"title":.*"Cowbird REST API".*\}'
+            }
+        }
+    }
+}
+
+# vi: tabstop=8 expandtab shiftwidth=4 softtabstop=4 syntax=python

birdhouse/components/cowbird/config/proxy/docker-compose-extra.yml

@@ -5,5 +5,12 @@ services:
   proxy:
     volumes:
       - ./components/cowbird/config/proxy/conf.extra-service.d:/etc/nginx/conf.extra-service.d/cowbird:ro
+      # NOTE:
+      #   Prefix '0_' to the mounted file name to ensure it is loaded first by 'birdhouse/config/canarie-api/docker_configuration.py'
+      #   This ensures that the alphabetical loading order it defines will first load the default configs, then will load
+      #   the Cowbird config, and then all python configuration within 'optional-components/canarie-api-full-monitoring'.
+      #   It is important to load Cowbird before, as the full monitoring need to know if 'Cowbird' service is added to the
+      #   stack in order to apply the corresponding public endpoint conditionally.
+      - ./components/cowbird/config/proxy/canarie_api_monitoring.py:${CANARIE_MONITORING_EXTRA_CONF_DIR}/0_cowbird_config.py:ro
     links:
       - cowbird

birdhouse/components/weaver/config/canarie-api/canarie_api_monitoring.py.template

@@ -1,9 +1,38 @@
-SERVICES['node']['monitoring'].update({
-    'Weaver': {
-        'request': {
-            'url': 'https://${PAVICS_FQDN_PUBLIC}/weaver'
-        },
+SERVICES['Weaver'] = {
+    'info': {
+        'name': 'Weaver',
+        'synopsis': 'Weaver is an OGC-API flavored Execution Management Service (EMS) that allows the execution of workflows chaining various applications and Web Processing Services (WPS) inputs and outputs. Remote execution is deferred by the EMS to one or many Application, Deployment and Execution Service (ADES) or remote service providers, and employs Common Workflow Language (CWL) configurations to define an OGC Application Package deployed for each process.',
+        'version': "${WEAVER_VERSION}",
+        'institution': 'CRIM',
+        'releaseTime': get_release_time_from_repo_tag("github", "crim-ca/weaver", "${WEAVER_VERSION}"),
+        'researchSubject': 'Any',
+        'supportEmail': '${SUPPORT_EMAIL}',
+        'category': 'Processing',
+        'tags': ['Workflow', 'Processes', 'Deployment', 'Distribution', 'EMS', 'ADES', 'CWL', 'OGC']
     },
-})
+    'stats': {
+        'method': '.*',
+        'route': "/weaver/.*"
+    },
+    'redirect': {
+        'doc': 'https://pavics-weaver.readthedocs.io/',
+        'releasenotes': 'https://github.com/crim-ca/weaver/blob/master/CHANGES.rst',
+        'support': 'https://github.com/crim-ca/weaver/issues',
+        'source': 'https://github.com/crim-ca/weaver',
+        'tryme': 'https://${PAVICS_FQDN_PUBLIC}/weaver/',
+        'licence': 'https://github.com/crim-ca/weaver/blob/${WEAVER_VERSION}/LICENSE.txt',
+        'provenance': 'https://github.com/crim-ca/weaver'
+    },
+    "monitoring": {
+        "Weaver": {
+            'request': {
+                'url': 'http://weaver:4001/'
+            },
+            'response': {
+                'text': '\{.*"message":.*"Weaver Information".*\}'
+            }
+        }
+    }
+}
 
 # vi: tabstop=8 expandtab shiftwidth=4 softtabstop=4 syntax=python