Skip to content

Commit cf221e1

Browse files
committed
Merge branch 'master' into magpie-networking
2 parents 2c6a0d2 + c64ca91 commit cf221e1

File tree

93 files changed

+381
-356
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

93 files changed

+381
-356
lines changed

.bumpversion.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
[tool.bumpversion]
2-
current_version = "2.20.4"
2+
current_version = "2.21.0"
33
commit = true
44
tag = false
55
tag_name = "{new_version}"

CHANGES.md

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,109 @@
3636
Note that all of these configuration options will be ignored if `MAGPIE_NETWORK_ENABLED` is `false`.
3737

3838

39+
[2.21.0](https://github.com/bird-house/birdhouse-deploy/tree/2.21.0) (2026-01-27)
40+
------------------------------------------------------------------------------------------------------------------
41+
42+
## Changes
43+
44+
- Remove `proxy` component's dependency on `scheduler` and `scheduler-job-logrotate-nginx`
45+
46+
Creates new settings in `optional-components/proxy-log-volume` that create the `proxy-logs` docker volume as well
47+
as instructing Nginx to write access logs to an additional log file (specified by `PROXY_LOG_PATH`). These settings
48+
are included as a `COMPONENT_DEPENDENCY` in components that require access to the the `proxy` access logs as a
49+
regular file. If no components require access to these logs as a regular file then the `proxy` component will only
50+
write access logs to the stdout stream for that container.
51+
52+
Right now, the only components that require access to logs in this way are `components/canarie-api` and
53+
`optional-components/prometheus-log-parser`. Both of these now include `optional-components/proxy-log-volume` as a
54+
`COMPONENT_DEPENDENCY`.
55+
56+
Note: this means that if no optional components require `optional-components/proxy-log-volume` as a dependency
57+
then logs from the `proxy` container will only be written to stdout/stderr. This means that there is no need
58+
for any additional custom log rotation handling since the logs are handled directly by docker. This means that
59+
the `proxy` service itself no longer need to be dependant on the `scheduler` and `scheduler-job-logrotate-nginx`
60+
components.
61+
62+
Note: a previous discussion suggested that logs could be parsed directly from the stdout stream of the `proxy`
63+
container. However, there is no way to do so that doesn't require very hacky workarounds. Possible solutions that
64+
were explored and rejected include:
65+
66+
- Mounting the log file from the `proxy` container from the host to the relevant containers.
67+
Rejected because this practice is highly discouraged by docker as the actual storage location of log files
68+
is not standardized and may be changed in future versions.
69+
- Writing logs to a named pipe or socket within the `proxy` container.
70+
Rejected because this is very difficult to set up and is untested when then mounted to other containers.
71+
Also, a different named pipe would be required for each consumer which is currently very difficult to set up
72+
using birdhouse's deployment tools.
73+
74+
**Breaking Change**: if a custom component (not included in this repository) uses the `proxy-logs` named volume.
75+
It must now include `optional-components/proxy-log-volume` as a `COMPONENT_DEPENDENCY` for that custom component.
76+
77+
**Breaking Change**: if `SCHEDULER_JOB_BACKUP_ARGS` specifies `-l proxy` explicitly (not `-l '*'`) then this should
78+
be changed to `-l proxy-log-volume` since the backup script has been moved. Note that it is not necessary to
79+
specify `-l proxy-log-volume` if `--birdhouse-logs` is also specified because the log data is identical in both.
80+
81+
- Make docker compose logging options configurable
82+
83+
Introduce a new component `components/logging` that sets default logging options for all docker
84+
compose services started by `birdhouse-deploy`. This component is enabled by default.
85+
86+
The default value is set by the `BIRDHOUSE_DOCKER_LOGGING_DEFAULT` environment variable. To change the default
87+
value, set the `BIRDHOUSE_DOCKER_LOGGING_DEFAULT` to a JSON string in the local environment file that contains
88+
a valid
89+
[docker compose logging configuration](https://docs.docker.com/reference/compose-file/services/#logging).
90+
91+
For example, to set the default driver to "local" set the following in your local environment file:
92+
93+
```sh
94+
export BIRDHOUSE_DOCKER_LOGGING_DEFAULT='{"driver": "local"}'
95+
```
96+
97+
You can also override logging options for a single service using environment variables using a variable
98+
`BIRDHOUSE_DOCKER_LOGGING_<service_name>` where `<service_name>` is the uppercase name of the docker compose
99+
service with hyphens replaced with underscores. For example, to set the default driver to "local" only for the
100+
`weaver-worker` service:
101+
102+
```sh
103+
export BIRDHOUSE_DOCKER_LOGGING_WEAVER_WORKER='{"driver": "local"}'
104+
```
105+
106+
Logging options can can also be set directly in a component's ``docker-compose-extra.yml`` file.
107+
The order of precedence for logging options are as follows:
108+
109+
1. logging options specified by `BIRDHOUSE_DOCKER_LOGGING_<service_name>` environment variable
110+
2. logging options set in a `docker-compose-extra.yml` file
111+
3. logging options specified by `BIRDHOUSE_DOCKER_LOGGING_DEFAULT` environment variable
112+
113+
- Add script that automatically updates postgres databases to a later version
114+
115+
In anticipation of upgrading postgres databases in the future, this introduces a script that automatically
116+
upgrades postgres databases using the backup/restore process.
117+
118+
This includes magpie and all WPS birds that use the postgres component. This does not include test component
119+
like `optional-components/generic_bird` and will not update custom components (ones not from this repository).
120+
121+
Test components are not assumed to have persistent data that needs to be updated and we cannot guarantee that
122+
other postgres databases used by components outside this repository do not require additional steps (data
123+
migrations) in order to comply with a different version of postgres.
124+
125+
It will update postgres databases to the version specified by the `POSTGRES_VERSION_UPDATE` environment variable.
126+
All of the old database files will be copied to a temporary directory in case you want to inspect them or revert
127+
this operation later on. To specify which directory to write these backups to set the `BIRDHOUSE_BACKUP_DATA_DIR` variable
128+
(default: `${TMPDIR:-/tmp}/birdhouse-postgres-migrate-backup/`)
129+
130+
Note that backups in the form of database dumps will also be written to the named volume or directory specified
131+
by the `BIRDHOUSE_BACKUP_VOLUME` variable.
132+
133+
For example, to update the current postgres databases to version 18.1 and write backups to `/tmp/test/`
134+
135+
```sh
136+
$ POSTGRES_VERSION_UPDATE=18.1 BIRDHOUSE_BACKUP_DATA_DIR=/tmp/test/ birdhouse/scripts/update-postgresh.sh
137+
```
138+
139+
In a future update we can update the postgres versions and tell users to run this script first in order to safely
140+
migrate data from one version to the next.
141+
39142
[2.20.4](https://github.com/bird-house/birdhouse-deploy/tree/2.20.4) (2026-01-16)
40143
------------------------------------------------------------------------------------------------------------------
41144

Makefile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ override BIRDHOUSE_MAKE_DIR := $(shell realpath -P $$(dirname $(BIRDHOUSE_MAKE_C
88
# Generic variables
99
override SHELL := bash
1010
override APP_NAME := birdhouse-deploy
11-
override APP_VERSION := 2.20.4
11+
override APP_VERSION := 2.21.0
1212

1313
# utility to remove comments after value of an option variable
1414
override clean_opt = $(shell echo "$(1)" | $(_SED) -r -e "s/[ '$'\t'']+$$//g")

README.rst

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -18,13 +18,13 @@ for a full-fledged production platform.
1818
* - citation
1919
- | |citation|
2020

21-
.. |commits-since| image:: https://img.shields.io/github/commits-since/bird-house/birdhouse-deploy/2.20.4.svg
21+
.. |commits-since| image:: https://img.shields.io/github/commits-since/bird-house/birdhouse-deploy/2.21.0.svg
2222
:alt: Commits since latest release
23-
:target: https://github.com/bird-house/birdhouse-deploy/compare/2.20.4...master
23+
:target: https://github.com/bird-house/birdhouse-deploy/compare/2.21.0...master
2424

25-
.. |latest-version| image:: https://img.shields.io/badge/tag-2.20.4-blue.svg?style=flat
25+
.. |latest-version| image:: https://img.shields.io/badge/tag-2.21.0-blue.svg?style=flat
2626
:alt: Latest Tag
27-
:target: https://github.com/bird-house/birdhouse-deploy/tree/2.20.4
27+
:target: https://github.com/bird-house/birdhouse-deploy/tree/2.21.0
2828

2929
.. |readthedocs| image:: https://readthedocs.org/projects/birdhouse-deploy/badge/?version=latest
3030
:alt: ReadTheDocs Build Status (latest version)

RELEASE.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
2.20.4 2026-01-16T16:40:32Z
1+
2.21.0 2026-01-27T20:49:01Z

birdhouse/components/README.rst

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1040,3 +1040,36 @@ How to Enable the Component
10401040
---------------------------
10411041
10421042
- Do not enable this component directly. It will be enabled as a dependency of other components
1043+
1044+
logging
1045+
=======
1046+
1047+
Sets default logging options for all docker compose services started by `birdhouse-deploy`.
1048+
1049+
The default value is set by the ``BIRDHOUSE_DOCKER_LOGGING_DEFAULT`` environment variable. To change the default
1050+
value, set the ``BIRDHOUSE_DOCKER_LOGGING_DEFAULT`` to a JSON string in the local environment file that contains
1051+
a valid `docker compose logging configuration`_.
1052+
1053+
For example, to set the default driver to "local" set the following in your local environment file:
1054+
1055+
.. code::shell
1056+
1057+
export BIRDHOUSE_DOCKER_LOGGING_DEFAULT='{"driver": "local"}'
1058+
1059+
You can also override logging options for a single service using environment variables using a variable
1060+
``BIRDHOUSE_DOCKER_LOGGING_<service_name>`` where ``<service_name>`` is the uppercase name of the docker compose service
1061+
with hyphens replaced with underscores. For example, to set the default driver to "local" only for the ``weaver-worker``
1062+
service:
1063+
1064+
.. code::shell
1065+
1066+
export BIRDHOUSE_DOCKER_LOGGING_WEAVER_WORKER='{"driver": "local"}'
1067+
1068+
Logging options can can also be set directly in a component's ``docker-compose-extra.yml`` file. The order of
1069+
precedence for logging options are as follows:
1070+
1071+
1. logging options specified by ``BIRDHOUSE_DOCKER_LOGGING_<service_name>`` environment variable
1072+
2. logging options set in a ``docker-compose-extra.yml`` file
1073+
3. logging options specified by ``BIRDHOUSE_DOCKER_LOGGING_DEFAULT`` environment variable
1074+
1075+
.. _docker compose logging configuration: https://docs.docker.com/reference/compose-file/services/#logging

birdhouse/components/canarie-api/config/proxy/docker-compose-extra.yml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,3 @@ services:
22
proxy:
33
volumes:
44
- ./components/canarie-api/config/proxy/conf.extra-service.d:/etc/nginx/conf.extra-service.d/canarie-api:ro
5-
- proxy-logs:${PROXY_LOG_DIR}

birdhouse/components/canarie-api/default.env

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,5 +12,5 @@ export DELAYED_EVAL="
1212

1313
# add any component that this component requires to run
1414
COMPONENT_DEPENDENCIES="
15-
./components/proxy
15+
./optional-components/proxy-log-volume
1616
"

birdhouse/components/canarie-api/docker-compose-extra.yml

Lines changed: 1 addition & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,3 @@
1-
2-
x-logging:
3-
&default-logging
4-
driver: "json-file"
5-
options:
6-
max-size: "50m"
7-
max-file: "10"
8-
91
services:
102
canarie-api:
113
image: ${CANARIE_IMAGE}
@@ -19,7 +11,6 @@ services:
1911
CANARIE_MONITORING_EXTRA_CONF_DIR: ${CANARIE_MONITORING_EXTRA_CONF_DIR}
2012
restart: always
2113
entrypoint: /entrypoint
22-
logging: *default-logging
2314
healthcheck:
2415
test: ["CMD", "curl", "--fail", "http://localhost:2000"]
2516
interval: 60s
@@ -44,8 +35,7 @@ services:
4435
CANARIE_MONITORING_EXTRA_CONF_DIR: ${CANARIE_MONITORING_EXTRA_CONF_DIR}
4536
restart: always
4637
entrypoint: /entrypoint
47-
logging: *default-logging
4838

39+
# Note: proxy-logs volume is defined in the proxy-logs-volume component
4940
volumes:
5041
canarie-data:
51-
proxy-logs:

birdhouse/components/canarie-api/docker_configuration.py.template

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -108,8 +108,8 @@ SERVICES = {
108108
# NOTE:
109109
# Below version and release time auto-managed by 'make VERSION=x.y.z bump'.
110110
# Do NOT modify it manually. See 'Tagging policy' in 'birdhouse/README.rst'.
111-
'version': '2.20.4',
112-
'releaseTime': '2026-01-16T16:40:32Z',
111+
'version': '2.21.0',
112+
'releaseTime': '2026-01-27T20:49:01Z',
113113
'institution': '${BIRDHOUSE_INSTITUTION}',
114114
'researchSubject': '${BIRDHOUSE_SUBJECT}',
115115
'supportEmail': '${BIRDHOUSE_SUPPORT_EMAIL}',
@@ -141,8 +141,8 @@ PLATFORMS = {
141141
# NOTE:
142142
# Below version and release time auto-managed by 'make VERSION=x.y.z bump'.
143143
# Do NOT modify it manually. See 'Tagging policy' in 'birdhouse/README.rst'.
144-
'version': '2.20.4',
145-
'releaseTime': '2026-01-16T16:40:32Z',
144+
'version': '2.21.0',
145+
'releaseTime': '2026-01-27T20:49:01Z',
146146
'institution': '${BIRDHOUSE_INSTITUTION}',
147147
'researchSubject': '${BIRDHOUSE_SUBJECT}',
148148
'supportEmail': '${BIRDHOUSE_SUPPORT_EMAIL}',

0 commit comments

Comments
 (0)