Make docker compose logging options configurable (#636)

## Overview

Introduce a new component `components/logging` that sets default logging
options for all docker compose services started by `birdhouse-deploy`.
This component is enabled by default.

The default value is set by the `BIRDHOUSE_LOGGING_DEFAULT` environment
variable. To change the default value, set the
`BIRDHOUSE_LOGGING_DEFAULT` to a JSON string in the local environment
file that contains a valid [docker compose logging
configuration](https://docs.docker.com/reference/compose-file/services/#logging).

For example, to set the default driver to "local" set the following in
your local environment file:

```sh
export BIRDHOUSE_LOGGING_DEFAULT='{"driver": "local"}'
```

You can also override logging options for a single service using
environment variables using a variable
`BIRDHOUSE_LOGGING_<service_name>` where `<service_name>` is the
uppercase name of the docker compose service with hyphens replaced with
underscores. For example, to set the default driver to "local" only for
the `weaver-worker` service:

```sh
export BIRDHOUSE_LOGGING_WEAVER_WORKER='{"driver": "local"}'
```

Logging options can can also be set directly in a component's
``docker-compose-extra.yml`` file.
The order of precedence for logging options are as follows:

1. logging options specified by `BIRDHOUSE_LOGGING_<service_name>`
environment variable
2. logging options set in a `docker-compose-extra.yml` file
3. logging options specified by `BIRDHOUSE_LOGGING_DEFAULT` environment
variable

## Changes

**Non-breaking changes**
- Adds new required component

**Breaking changes**
- None

## Related Issue / Discussion

- Initially from:
#631 (comment)

## Additional Information

## CI Operations

<!--
The test suite can be run using a different DACCS config with
``birdhouse_daccs_configs_branch: branch_name`` in the PR description.
To globally skip the test suite regardless of the commit message use
``birdhouse_skip_ci`` set to ``true`` in the PR description.

Using ``[<cmd>]`` (with the brackets) where ``<cmd> = skip ci`` in the
commit message will override ``birdhouse_skip_ci`` from the PR
description.
Such commit command can be used to override the PR description behavior
for a specific commit update.
However, a commit message cannot 'force run' a PR which the description
turns off the CI.
To run the CI, the PR should instead be updated with a ``true`` value,
and a running message can be posted in following PR comments to trigger
tests once again.
-->

birdhouse_daccs_configs_branch: master
birdhouse_skip_ci: false

Author: mishaschwartz

CHANGES.md

@@ -17,6 +17,38 @@
 
 ## Changes
 
+- Make docker compose logging options configurable
+
+  Introduce a new component `components/logging` that sets default logging options for all docker 
+  compose services started by `birdhouse-deploy`. This component is enabled by default.
+
+  The default value is set by the `BIRDHOUSE_DOCKER_LOGGING_DEFAULT` environment variable. To change the default
+  value, set the `BIRDHOUSE_DOCKER_LOGGING_DEFAULT` to a JSON string in the local environment file that contains
+  a valid 
+  [docker compose logging configuration](https://docs.docker.com/reference/compose-file/services/#logging).
+
+  For example, to set the default driver to "local" set the following in your local environment file:
+
+  ```sh
+  export BIRDHOUSE_DOCKER_LOGGING_DEFAULT='{"driver": "local"}'
+  ```
+
+  You can also override logging options for a single service using environment variables using a variable
+  `BIRDHOUSE_DOCKER_LOGGING_<service_name>` where `<service_name>` is the uppercase name of the docker compose 
+  service with hyphens replaced with underscores. For example, to set the default driver to "local" only for the 
+  `weaver-worker` service:
+
+  ```sh
+  export BIRDHOUSE_DOCKER_LOGGING_WEAVER_WORKER='{"driver": "local"}'
+  ```
+
+  Logging options can can also be set directly in a component's ``docker-compose-extra.yml`` file. 
+  The order of precedence for logging options are as follows:
+
+  1. logging options specified by `BIRDHOUSE_DOCKER_LOGGING_<service_name>` environment variable
+  2. logging options set in a `docker-compose-extra.yml` file
+  3. logging options specified by `BIRDHOUSE_DOCKER_LOGGING_DEFAULT` environment variable
+
 - Add script that automatically updates postgres databases to a later version
 
   In anticipation of upgrading postgres databases in the future, this introduces a script that automatically

birdhouse/components/README.rst

@@ -1040,3 +1040,36 @@ How to Enable the Component
 ---------------------------
 
 - Do not enable this component directly. It will be enabled as a dependency of other components
+
+logging
+=======
+
+Sets default logging options for all docker compose services started by `birdhouse-deploy`.
+
+The default value is set by the ``BIRDHOUSE_DOCKER_LOGGING_DEFAULT`` environment variable. To change the default
+value, set the ``BIRDHOUSE_DOCKER_LOGGING_DEFAULT`` to a JSON string in the local environment file that contains
+a valid `docker compose logging configuration`_.
+
+For example, to set the default driver to "local" set the following in your local environment file:
+
+.. code::shell
+
+  export BIRDHOUSE_DOCKER_LOGGING_DEFAULT='{"driver": "local"}'
+
+You can also override logging options for a single service using environment variables using a variable
+``BIRDHOUSE_DOCKER_LOGGING_<service_name>`` where ``<service_name>`` is the uppercase name of the docker compose service 
+with hyphens replaced with underscores. For example, to set the default driver to "local" only for the ``weaver-worker``
+service:
+
+.. code::shell
+
+  export BIRDHOUSE_DOCKER_LOGGING_WEAVER_WORKER='{"driver": "local"}'
+
+Logging options can can also be set directly in a component's ``docker-compose-extra.yml`` file. The order of
+precedence for logging options are as follows:
+
+1. logging options specified by ``BIRDHOUSE_DOCKER_LOGGING_<service_name>`` environment variable
+2. logging options set in a ``docker-compose-extra.yml`` file
+3. logging options specified by ``BIRDHOUSE_DOCKER_LOGGING_DEFAULT`` environment variable
+
+.. _docker compose logging configuration: https://docs.docker.com/reference/compose-file/services/#logging

birdhouse/components/canarie-api/docker-compose-extra.yml

@@ -1,11 +1,3 @@
-
-x-logging:
-  &default-logging
-  driver: "json-file"
-  options:
-    max-size: "50m"
-    max-file: "10"
-
 services:
   canarie-api:
     image: ${CANARIE_IMAGE}
@@ -19,7 +11,6 @@ services:
       CANARIE_MONITORING_EXTRA_CONF_DIR: ${CANARIE_MONITORING_EXTRA_CONF_DIR}
     restart: always
     entrypoint: /entrypoint
-    logging: *default-logging
     healthcheck:
       test: ["CMD", "curl", "--fail", "http://localhost:2000"]
       interval: 60s
@@ -44,7 +35,6 @@ services:
       CANARIE_MONITORING_EXTRA_CONF_DIR: ${CANARIE_MONITORING_EXTRA_CONF_DIR}
     restart: always
     entrypoint: /entrypoint
-    logging: *default-logging
 
 volumes:
   canarie-data:

birdhouse/components/cowbird/config/canarie-api/docker-compose-extra.yml

@@ -1,4 +1,3 @@
-
 services:
   canarie-api: &canarie-volumes
     volumes:

birdhouse/components/cowbird/config/geoserver/docker-compose-extra.yml

@@ -1,4 +1,3 @@
-
 services:
   cowbird:
     environment:

birdhouse/components/cowbird/config/magpie/docker-compose-extra.yml

@@ -1,4 +1,3 @@
-
 services:
   # extend Magpie permissions to grant access to Cowbird API via secured Twitcher proxy
   magpie:

birdhouse/components/cowbird/config/proxy/docker-compose-extra.yml

@@ -1,4 +1,3 @@
-
 services:
   # extend proxy with endpoint and config for Cowbird API access
   proxy:

birdhouse/components/cowbird/docker-compose-extra.yml

@@ -1,11 +1,3 @@
-
-x-logging:
-  &default-logging
-  driver: "json-file"
-  options:
-    max-size: "50m"
-    max-file: "10"
-
 services:
   cowbird:
     image: ${COWBIRD_IMAGE_API}
@@ -43,7 +35,6 @@ services:
       - ./components/cowbird/config/cowbird/celeryconfig.py:/opt/local/src/cowbird/config/celeryconfig.py
       - "${BIRDHOUSE_DATA_PERSIST_SHARED_ROOT}:${BIRDHOUSE_DATA_PERSIST_SHARED_ROOT}"
     restart: always
-    logging: *default-logging
     healthcheck:
       test: ["CMD", "wget", "-qO-", "http://0.0.0.0:7000"]
       interval: 60s
@@ -79,7 +70,6 @@ services:
       - ./components/cowbird/config/cowbird/celeryconfig.py:/opt/local/src/cowbird/config/celeryconfig.py
       - "${BIRDHOUSE_DATA_PERSIST_SHARED_ROOT}/${USER_WORKSPACES}:${BIRDHOUSE_DATA_PERSIST_SHARED_ROOT}/${USER_WORKSPACES}"
     restart: always
-    logging: *default-logging
     # TODO: create a stable healthcheck for this service (`inspect ping` or `status` is not reliable)
 
   # Dedicated database for Cowbird, since other 'mongodb' image does not employ the same version.
@@ -91,7 +81,6 @@ services:
     volumes:
       - ${COWBIRD_MONGODB_DATA_DIR}:/data/db
     restart: always
-    logging: *default-logging
     healthcheck:
       test: ["CMD", "mongo", "--eval", "print('connected')"]
       interval: 60s

birdhouse/components/dggs/config/magpie/docker-compose-extra.yml

@@ -1,4 +1,3 @@
-
 services:
   magpie:
     volumes:

birdhouse/components/dggs/config/twitcher/docker-compose-extra.yml

@@ -1,4 +1,3 @@
-
 services:
   # extend twitcher with MagpieAdapter hooks employed for DGGS proxied requests
   twitcher: