Add OGC API DGGS service + associated utility components

CHANGES.md

@@ -15,7 +15,23 @@
 [Unreleased](https://github.com/bird-house/birdhouse-deploy/tree/master) (latest)
 ------------------------------------------------------------------------------------------------------------------
 
-[//]: # (list changes here, using '-' for each new entry, remove this when items are added)
+## Changes
+
+- DGGS: Add the new `components/dggs` providing an OGC API for Discrete Global Grid Systems.
+  - DGGS API available through `/dggs-api` path (default, configurable via `DGGS_API_PATH`).
+  - Redirects available for `/ogcapi/dggs/...` and  `/ogcapi/collections/.../dggs/...`.
+  - Sample configuration (minimum 1 resolvable data provider required) uses the new
+    feature of `optional-components/secure-data-proxy` on CRIM's Hirondelle server.
+
+- Data: Allow `optional-components/secure-data-proxy` to define generic and flexible locations.
+  - `SECURE_DATA_PROXY_ROOT` can be defined as mount directory inside the `proxy` service.
+  - `SECURE_DATA_PROXY_LOCATIONS` can be defined with any amount of custom locations.
+  - All locations can be configured (as desired) under Magpie `secure-data-proxy` service for access control.
+  - Other components (`wps_output-volume`, `stac-data-proxy`) that can optionally use this security middleware
+    via `SECURE_DATA_PROXY_AUTH_INCLUDE` can still do so. Their mount points are handled separately.
+
+- Weaver: Modified `/ogcapi/...` redirections strategy via `WEAVER_ALT_PREFIX_PROXY_LOCATION`. 
+  - Allows other OGC APIs to reuse the same prefix to provide a unified location to access them.
 
 [2.18.11](https://github.com/bird-house/birdhouse-deploy/tree/2.18.11) (2025-11-13)
 ------------------------------------------------------------------------------------------------------------------

birdhouse/components/README.rst

@@ -720,14 +720,12 @@ How to Enable the Component
 - Edit ``env.local`` (a copy of `env.local.example`_)
 - Add ``./components/stac`` to ``BIRDHOUSE_EXTRA_CONF_DIRS``.
 
+
 STAC Browser
 ============
 
 STAC Browser is a web UI used to interact with the STAC API. 
 
-Usage
------
-
 The STAC API can be browsed via the ``stac-browser`` component. By default, the browser will point to the STAC API 
 exposed by the current ``components/stac`` service. 
 Once this component is enabled, the STAC browser will be available at the ``https://<BIRDHOUSE_FQDN_PUBLIC>/stac-browser`` 
@@ -748,12 +746,53 @@ For example:
   # you can update it to
   export STAC_CORS_ORIGINS='http://example.com ~http:(www|other)\.api\.example\.com https://geojson.io'
 
+
 How to Enable the Component
 ---------------------------
 
 - Edit ``env.local`` (a copy of `env.local.example`_)
 - Add ``./components/stac-browser`` to ``BIRDHOUSE_EXTRA_CONF_DIRS``.
 
+
+.. _components_dggs:
+
+DGGS: Discrete Global Grid Systems API
+======================================
+
+`DGGS`_ is a spatial reference system combining a discrete global grid hierarchy with a zone identifier, in contrast
+to typical ``(lat, lon)`` spatial reference systems. By using a predefined and deterministic order of zone IDs and
+refinement sub-zones, DGGS enables efficient access, representation and analysis of spatial data that has been
+quantized over a certain grid reference system (DGGRS).
+
+The *OGC API - DGGS* definition implemented by this service is a RESTful API that provides access to DGGS resources,
+definitions, zonal query conversion, and data retrieval from precomputed datasets.
+
+.. _DGGS: https://ogcapi.ogc.org/dggs/
+
+Usage
+-----
+
+Once enabled, the API will be accessible (by default) on the ``/dggs-api`` endpoint.
+It will also be available through the common ``/ogcapi/dggs`` alias.
+
+Refer to the `DGGS`_ OGC API documentation for specific endpoints and features.
+
+Refer to `vgrid DGGS <https://github.com/opengeoshub#vgrid-dggs>`_ and
+the `vgrid repository <https://github.com/opengeoshub/vgrid>`_ for a relatively extensive
+collection of DGGS tools and its associated data manipulation ecosystem (using ``xarray``, QGIS plugin, etc.).
+
+How to Enable the Component
+---------------------------
+
+- Edit ``env.local`` (a copy of `env.local.example`_)
+- Add ``./components/dggs`` to ``BIRDHOUSE_EXTRA_CONF_DIRS``.
+- Define ``DGGS_CONFIG_PATH`` in the ``env.local`` with custom definitions.
+  Alternatively, employ sample DGGS dataset and configuration by enabling ``./optional-components/dggs-data-sample``.
+  Enabling this optional component will set ``DGGS_CONFIG_PATH`` with a predefined configuration for this sample data.
+  See the `PyDGGS-API implementation <https://github.com/LandscapeGeoinformatics/pydggsapi>`_ for more details.
+- Optionally, configure variables in ``./components/dggs/default.env`` via ``env.local`` to customize the service.
+
+
 Canarie-API
 ===========
 
@@ -986,10 +1025,11 @@ of all processes executed by these services.
 Usage
 -----
 
-All outputs from these processes will become available at the ``${BIRDHOUSE_PROXY_SCHEME}://${BIRDHOUSE_FQDN_PUBLIC}/wpsoutputs`` endpoint.
+All outputs from these processes will become available at
+the ``${BIRDHOUSE_PROXY_SCHEME}://${BIRDHOUSE_FQDN_PUBLIC}/wpsoutputs`` endpoint.
 
 By default, this endpoint is not protected. To secure access to this endpoint it is highly recommended to enable the
-`./optional-components/secure-data-proxy` component as well.
+``./optional-components/secure-data-proxy`` component as well.
 
 How to Enable the Component
 ---------------------------

birdhouse/components/dggs/.gitignore

@@ -0,0 +1,3 @@
+config/magpie/config.yml
+config/proxy/conf.extra-service.d/dggs.conf
+service-config.json

birdhouse/components/dggs/config/magpie/config.yml.template

@@ -0,0 +1,91 @@
+providers:
+  # definition of STAC service for API access
+  ${DGGS_API_NAME}:
+    url: http://dggs:8000
+    title: Discrete Global Grid System (DGGS) Service
+    public: true
+    c4i: false
+    type: api
+    sync_type: api
+
+  ${DGGS_TILES_NAME}:
+    url: http://dggs:8000
+    title: Tiles support over Discrete Global Grid System (DGGS) Service
+    public: true
+    c4i: false
+    type: api
+    sync_type: api
+
+
+permissions:
+  - service: ${DGGS_API_NAME}
+    resource: /openapi.json
+    permission: read
+    group: anonymous
+    action: create
+  - service: ${DGGS_API_NAME}
+    resource: ${DGGS_DOCS_PATH_INTERNAL}
+    permission: read
+    group: anonymous
+    action: create
+  - service: ${DGGS_API_NAME}
+    resource: /docs
+    permission: read
+    group: anonymous
+    action: create
+  - service: ${DGGS_API_NAME}
+    resource: /api
+    permission: read
+    group: anonymous
+    action: create
+
+  # === DGGS API ===
+  # /${DGGS_API_NAME}/dggs/...
+  # /${DGGS_API_NAME}/collections/{collectionId}/dggs/...
+
+  # Administrator permissions over whole service
+  - service: ${DGGS_API_NAME}
+    permission: read
+    group: administrators
+    action: create
+  - service: ${DGGS_API_NAME}
+    permission: write
+    group: administrators
+    action: create
+  # Open access to the entrypoint
+  - service: ${DGGS_API_NAME}
+    permission: read-match
+    group: anonymous
+    action: create
+  # Open access to the generic DGGRS definitions
+  # DGGS-nested endpoints and Collections access managed by user/groups.
+  - service: ${DGGS_API_NAME}
+    resource: /dggs
+    permission: read-match
+    group: anonymous
+    action: create
+  # Open access to the documentation endpoints
+  - service: ${DGGS_API_NAME}
+    resource: /conformance
+    permission: read-match
+    group: anonymous
+    action: create
+
+  # === Tiles API ===
+  # /${DGGS_TILES_NAME}/{collectionId}.json
+  # /${DGGS_TILES_NAME}/{collectionId}/{z}/{x}/{x}
+
+  # Administrator permissions over whole service
+  - service: ${DGGS_TILES_NAME}
+    permission: read
+    group: administrators
+    action: create
+  - service: ${DGGS_TILES_NAME}
+    permission: write
+    group: administrators
+    action: create
+  # Tiles-only endpoint. Collections access managed by user/groups.
+  - service: ${DGGS_TILES_NAME}
+    permission: read-match
+    group: anonymous
+    action: create

birdhouse/components/dggs/config/magpie/docker-compose-extra.yml

@@ -0,0 +1,6 @@
+
+services:
+  magpie:
+    volumes:
+      - ./components/dggs/config/magpie/config.yml:${MAGPIE_PROVIDERS_CONFIG_PATH}/dggs.yml:ro
+      - ./components/dggs/config/magpie/config.yml:${MAGPIE_PERMISSIONS_CONFIG_PATH}/dggs.yml:ro

birdhouse/components/dggs/config/proxy/conf.extra-service.d/dggs.conf.template

@@ -0,0 +1,120 @@
+
+    location = /dggs-auth {
+        internal;
+        # note: using 'TWITCHER_VERIFY_PATH' path to avoid performing the request via proxy 'TWITCHER_PROTECTED_PATH'
+        # This ensures that access is validated for the user, but does not trigger its access/download twice.
+        # It is also more efficient, since less contents are transferred/buffered.
+        proxy_pass ${BIRDHOUSE_PROXY_SCHEME}://${BIRDHOUSE_FQDN_PUBLIC}${TWITCHER_VERIFY_PATH}/$request_uri;
+        proxy_pass_request_body off;
+        proxy_set_header Host $host;
+        proxy_set_header Content-Length "";
+        proxy_set_header X-Original-URI $request_uri;
+        proxy_set_header X-Forwarded-Proto $real_scheme;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Host $host:$server_port;
+    }
+
+    # Generic API docs
+    location ${DGGS_API_PATH}/docs {
+        auth_request /dggs-auth;
+        auth_request_set $auth_status $upstream_status;
+
+        proxy_pass http://dggs:8000${DGGS_DOCS_PATH_INTERNAL}/$is_args$args;
+        proxy_set_header Host $host;
+        #proxy_set_header X-Original-URI $request_uri;
+        # proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $real_scheme;
+        proxy_set_header X-Forwarded-Host $http_host:$server_port;
+        #proxy_set_header X-Script-Name ${DGGS_API_PATH};
+    }
+    location ${DGGS_TILES_PATH}/docs {
+        auth_request /dggs-auth;
+        auth_request_set $auth_status $upstream_status;
+
+        proxy_pass http://dggs:8000${DGGS_DOCS_PATH_INTERNAL}/$is_args$args;
+        proxy_set_header Host $host;
+        #proxy_set_header X-Original-URI $request_uri;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $real_scheme;
+        proxy_set_header X-Forwarded-Host $http_host:$server_port;
+        #proxy_set_header X-Script-Name ${DGGS_API_PATH};
+    }
+    # '/api' is the standard path by OGC: https://ogcapi.ogc.org/common/overview.html
+    location = ${DGGS_API_PATH}/api {
+        auth_request /dggs-auth;
+        auth_request_set $auth_status $upstream_status;
+
+        rewrite ^${DGGS_API_PATH}/(.*) ${DGGS_API_PATH_INTERNAL}/$1 break;
+        proxy_pass http://dggs:8000;
+        proxy_set_header Host $host;
+        proxy_set_header X-Original-URI $request_uri;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $real_scheme;
+        proxy_set_header X-Forwarded-Host $http_host:$server_port;
+        #proxy_set_header X-Script-Name ${DGGS_API_PATH};
+    }
+
+    # DGGS API
+    location = ${DGGS_API_PATH} {
+        return 308 ${DGGS_API_PATH}/$is_args$args;
+    }
+    location ~ ^${DGGS_API_PATH}/(.*)$ {
+        auth_request /dggs-auth;
+        auth_request_set $auth_status $upstream_status;
+
+        proxy_pass http://dggs:8000${DGGS_API_PATH_INTERNAL}/$1$is_args$args;
+        proxy_set_header Host $host;
+        proxy_set_header X-Original-URI $request_uri;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $real_scheme;
+        proxy_set_header X-Forwarded-Host $http_host:$server_port;
+    }
+    # Tiles API
+    location = ${DGGS_TILES_PATH} {
+        return 308 ${DGGS_TILES_PATH}/$is_args$args;
+    }
+    location ~ ^${DGGS_TILES_PATH}/(.*)$ {
+        auth_request /dggs-auth;
+        auth_request_set $auth_status $upstream_status;
+
+        proxy_pass http://dggs:8000${DGGS_TILES_PATH_INTERNAL}/$1$is_args$args;
+        proxy_set_header Host $host;
+        proxy_set_header X-Original-URI $request_uri;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $real_scheme;
+        proxy_set_header X-Forwarded-Host $http_host:$server_port;
+    }
+
+    # NOTE:
+    #   Redirect to internal network of Twitcher with root endpoint and alias allows to set
+    #   the same 'magpie' permissions on the 'dggs' service.
+    #   This allows verification of the same service user/group permissions references regardless
+    #   whether the *shortcut* endpoint, the alias or the explicit 'twitcher' proxy route is used.
+    location = ${TWITCHER_PROTECTED_PATH}${DGGS_API_PATH} {
+        return 308 ${DGGS_API_PATH}/$is_args$args;
+    }
+    location ~ ^${TWITCHER_PROTECTED_PATH}${DGGS_API_PATH}/(.*)$ {
+        return 308 ${DGGS_API_PATH}/$1$is_args$args;
+    }
+    location = ${TWITCHER_PROTECTED_PATH}${DGGS_TILES_PATH} {
+        return 308 ${DGGS_TILES_PATH}/$is_args$args;
+    }
+    location ~ ^${TWITCHER_PROTECTED_PATH}${DGGS_TILES_PATH}/(.*)$ {
+        return 308 ${DGGS_TILES_PATH}/$1$is_args$args;
+    }
+
+    # NOTE:
+    #   Redirect to typical endpoints employed by clients.
+    #   Use 307 instead of 308 as above to indicate and hint clients that this is only a
+    #   shortcut to be revalidatd rather than the definitive location of the actual service.
+    #   The 'collection' variant is explicitly listed since '/ogcapi/' prefix could be
+    #   used for other services (e.g.: '/ogcapi/processes' for Weaver).
+    location ~ ^/ogcapi/dggs(.*)$ {
+        return 307 ${DGGS_API_PATH}$1$is_args$args;
+    }
+    location ~ ^/ogcapi/collections/(.*)/dggs(.*)$ {
+        return 307 ${DGGS_API_PATH}/collections/$1/dggs$2$is_args$args;
+    }
+    location ~ ^/ogcapi/tiles(.*)$ {
+        return 307 ${DGGS_TILES_PATH}$1$is_args$args;
+    }

birdhouse/components/dggs/config/proxy/docker-compose-extra.yml

@@ -0,0 +1,4 @@
+services:
+  proxy:
+    volumes:
+    - ./components/dggs/config/proxy/conf.extra-service.d:/etc/nginx/conf.extra-service.d/dggs:ro

birdhouse/components/dggs/config/twitcher/docker-compose-extra.yml

@@ -0,0 +1,8 @@
+
+services:
+  # extend twitcher with MagpieAdapter hooks employed for DGGS proxied requests
+  twitcher:
+    volumes:
+      # NOTE: MagpieAdapter hooks are defined within Magpie config, but it is actually Twitcher proxy that runs them
+      # target mount location depends on 'MAGPIE_PROVIDERS_CONFIG_PATH' environment variable that is found under `birdhouse/components/twitcher/docker-compose-extra.yml`
+      - ./components/dggs/config/magpie/config.yml:/opt/birdhouse/src/magpie/config/dggs-config.yml:ro