Merge pull request #121 from stac-utils/multiWMTSLayers

Support multiple Layers in WMTS endpoint

.github/workflows/ci.yml

@@ -88,7 +88,7 @@ jobs:
 
       - name: Install python dependencies
         run: |
-          python -m pip install pypgstac==${{ env.PGSTAC_VERSION }} psycopg[pool] httpx pytest pytest-benchmark
+          python -m pip install pypgstac==${{ env.PGSTAC_VERSION }} psycopg[pool] httpx pytest pytest-benchmark rasterio
 
       - name: Ingest Stac Items/Collection
         run: |

CHANGES.md

@@ -1,8 +1,39 @@
 # Release Notes
 
-## 0,5,2 (TDB)
-
-* add `tilejson` URL links for `defaults layers` defined in mosaic's metadata in `/mosaic/register` and `/mosaic/{mosaic_id}/info` response
+## 0.6.0 (TDB)
+
+* add `tilejson` URL links for `layers` defined in mosaic's metadata in `/mosaic/register` and `/mosaic/{mosaic_id}/info` response
+* support multiple `layers` in `/mosaic/{mosaic_id}/WMTSCapabilities.xml` endpoint created from mosaic's metadata
+
+**breaking change**
+
+* In `/mosaic/WMTSCapabilities.xml` we removed the query-parameters related to the `tile` endpoint (which are forwarded) so `?assets=` is no more required.
+The endpoint will still raise an error if there are no `layers` in the mosaic metadata and no required tile's parameters are passed.
+
+    ```python
+    # before
+    response = httpx.get("/mosaic/{mosaic_id}/WMTSCapabilities.xml")
+    assert response.status_code == 400
+
+    response = httpx.get("/mosaic/{mosaic_id}/WMTSCapabilities.xml?assets=cog")
+    assert response.status_code == 200
+
+    # now
+    # If the mosaic has `defaults` layers set in the metadata
+    # we will construct a WMTS document with multiple layers, so no need for the user to pass any `assets=`
+    response = httpx.get("/mosaic/{mosaic_id}/WMTSCapabilities.xml")
+    assert response.status_code == 200
+    with rasterio.open(io.BytesIO(response.content)) as src:
+        assert src.profile["driver"] == "WMTS"
+        assert len(src.subdatasets) == 2
+
+    # If the user pass any valid `tile` parameters, an additional layer will be added to the one from the metadata
+    response = httpx.get("/mosaic/{mosaic_id}/WMTSCapabilities.xml?assets=cog")
+    assert response.status_code == 200
+    with rasterio.open(io.BytesIO(response.content)) as src:
+        assert src.profile["driver"] == "WMTS"
+        assert len(src.subdatasets) == 3
+    ```
 
 ## 0.5.1 (2023-08-03)
 

docs/src/intro.md

@@ -10,7 +10,9 @@ By default the main application (`titiler.pgstac.main.app`) provides two sets of
 
 ## Mosaic
 
-#### 1. Register a `Search` request (Mosaic)
+The goal of the `mosaic` endpoints is to use any [`search`](https://github.com/radiantearth/stac-api-spec/tree/master/item-search) query to create tiles. `titiler-pgstac` provides a set of endpoint to `register` and `list` the `search` queries.
+
+### Register a `Search` request
 
 ![](https://user-images.githubusercontent.com/10407788/132193537-0560016f-09bc-4a25-8a2a-eac9b50bc28a.png)
 
@@ -70,16 +72,16 @@ curl -X 'POST' 'http://127.0.0.1:8081/mosaic/register' \
 }
 ```
 
-##### 1.1 Get Mosaic metadata
+##### Mosaic metadata
 
 ```bash
 curl http://127.0.0.1:8081/mosaic/5063721f06957d6b2320326d82e90d1e/info | jq
 
 >> {
   "search": {
-    "hash": "5063721f06957d6b2320326d82e90d1e",
-    "search": {
-      "filter": {
+    "hash": "5063721f06957d6b2320326d82e90d1e",  # <-- this is the search/mosaic ID
+    "search": {  # <-- Summary of the search request
+      "filter": {  # <-- this is CQL2 filter associated with the search
         "op": "and",
         "args": [
           {
@@ -129,12 +131,12 @@ curl http://127.0.0.1:8081/mosaic/5063721f06957d6b2320326d82e90d1e/info | jq
         ]
       }
     },
-    "_where": "(  ( (collection_id = 'landsat-c2l2-sr') and st_intersects(geometry, '0103000020E610000001000000050000000000000000F05EC055F6687D502741400000000000F05EC02D553EA94A6943400000000000885DC02D553EA94A6943400000000000885DC055F6687D502741400000000000F05EC055F6687D50274140'::geometry) )  )  ",
+    "_where": "(  ( (collection_id = 'landsat-c2l2-sr') and st_intersects(geometry, '0103000020E610000001000000050000000000000000F05EC055F6687D502741400000000000F05EC02D553EA94A6943400000000000885DC02D553EA94A6943400000000000885DC055F6687D502741400000000000F05EC055F6687D50274140'::geometry) )  )  ",  # <-- internal pgstac WHERE expression
     "orderby": "datetime DESC, id DESC",
-    "lastused": "2022-03-03T11:44:55.878504+00:00",
-    "usecount": 2,
-    "metadata": {
-      "type": "mosaic"
+    "lastused": "2022-03-03T11:44:55.878504+00:00",  # <-- internal pgstac variable
+    "usecount": 2,  # <-- internal pgstac variable
+    "metadata": {  # <-- titiler-pgstac Mosaic Metadata
+      "type": "mosaic"  # <-- where using the `/mosaic/register` endpoint, titiler-pgstac will add `type=mosaic` to the metadata
     }
   },
   "links": [
@@ -199,7 +201,7 @@ curl http://127.0.0.1:8081/mosaic/f31d7de8a5ddfa3a80b9a9dd06378db1/info | jq '.s
 }
 ```
 
-#### 2. Fetch mosaic `Tiles`
+### Fetch mosaic `Tiles`
 
 When we have a `searchid` we can now call the dynamic tiler and ask for Map Tiles.
 

docs/src/mosaic_endpoints.md

@@ -16,28 +16,29 @@ The `titiler.pgstac` package comes with a full FastAPI application with Mosaic a
 
 `:endpoint:/mosaic/register - [POST]`
 
-- **Body**: A valid STAC Search query (see: https://github.com/radiantearth/stac-api-spec/tree/master/item-search)
+- **Body** (a combination of Search+Metadata): A JSON body composed of a valid **STAC Search** query (see: https://github.com/radiantearth/stac-api-spec/tree/master/item-search) and Mosaic's metadata.
 
 ```json
 // titiler-pgstac search body example
 {
+  // STAC search query
   "collections": [
     "string"
   ],
   "ids": [
     "string"
   ],
   "bbox": [
-    "string",
-    "string",
-    "string",
-    "string"
+    "number",
+    "number",
+    "number",
+    "number"
   ],
   "intersects": {
     "type": "Point",
     "coordinates": [
-      "string",
-      "string"
+      "number",
+      "number"
     ]
   },
   "query": {
@@ -49,19 +50,21 @@ The `titiler.pgstac` package comes with a full FastAPI application with Mosaic a
   "datetime": "string",
   "sortby": "string",
   "filter-lang": "cql-json",
+  // titiler-pgstac mosaic metadata
   "metadata": {
     "type": "mosaic",
     "bounds": [
-      "string",
-      "string",
-      "string",
-      "string"
+      "number",
+      "number",
+      "number",
+      "number"
     ],
-    "minzoom": 0,
-    "maxzoom": 0,
+    "minzoom": "number",
+    "maxzoom": "number",
     "name": "string",
     "assets": [
-      "string"
+      "string",
+      "string",
     ],
     "defaults": {}
   }
@@ -271,32 +274,14 @@ Example:
     - **TileMatrixSetId**: TileMatrixSet name, default is `WebMercatorQuad`. OPTIONAL
 
 - QueryParams:
-    - **tile_format**: Output image format, default is set to None and will be either JPEG or PNG depending on masked value.
+    - **tile_format**: Output image format, default is set to PNG.
     - **tile_scale**: Tile size scale, default is set to 1 (256x256). OPTIONAL
     - **minzoom**: Overwrite default minzoom. OPTIONAL
     - **maxzoom**: Overwrite default maxzoom. OPTIONAL
-    - **expression** (str): rio-tiler's math expression with asset names (e.g `Asset1_b1/Asset2_b1`).
-    - **asset_as_band** (bool): tell rio-tiler that each asset is a 1 band dataset, so expression `Asset1/Asset2` can be passed.
-    - **asset_bidx** (array[str]): Per asset band math expression (e.g `Asset1|1;2;3`).
-    - **nodata** (str, int, float): Overwrite internal Nodata value.
-    - **unscale** (bool): Apply dataset internal Scale/Offset.
-    - **resampling** (str): rasterio resampling method. Default is `nearest`.
-    - **algorithm** (str): Custom algorithm name (e.g `hillshade`).
-    - **algorithm_params** (str): JSON encoded algorithm parameters.
-    - **rescale** (array[str]): Comma (',') delimited Min,Max range (e.g `rescale=0,1000`, `rescale=0,1000&rescale=0,3000&rescale=0,2000`).
-    - **color_formula** (str): rio-color formula.
-    - **colormap** (str): JSON encoded custom Colormap.
-    - **colormap_name** (str): rio-tiler color map name.
-    - **return_mask** (bool): Add mask to the output data. Default is True.
-    - **buffer** (float): Add buffer on each side of the tile (e.g 0.5 = 257x257, 1.0 = 258x258).
-    - **scan_limit** (int): Return as soon as we scan N items, Default is 10,000 in PgSTAC.
-    - **items_limit** (int): Return as soon as we have N items per geometry, Default is 100 in PgSTAC.
-    - **time_limit** (int): Return after N seconds to avoid long requests, Default is 5sec in PgSTAC.
-    - **exitwhenfull** (bool): Return as soon as the geometry is fully covered, Default is `True` in PgSTAC.
-    - **skipcovered** (bool): Skip any items that would show up completely under the previous items, Default is `True` in PgSTAC.
+
 
 !!! important
-    **assets** OR **expression** is required
+    additional query-parameters will be forwarded to the `tile` URL. If no `defaults` mosaic metadata, **assets** OR **expression** will be required
 
 Example:
 

pyproject.toml

@@ -122,3 +122,8 @@ no_implicit_optional = true
 strict_optional = true
 namespace_packages = true
 explicit_package_bases = true
+
+[tool.pytest.ini_options]
+filterwarnings = [
+    "ignore::rasterio.errors.NotGeoreferencedWarning",
+]

tests/test_mosaic.py

@@ -4,6 +4,7 @@
 from datetime import datetime
 from unittest.mock import patch
 
+import pytest
 import rasterio
 from rasterio.crs import CRS
 
@@ -487,9 +488,9 @@ def test_query_with_metadata(app):
     assert resp["searchid"]
     assert resp["links"]
 
-    cql2_id = resp["searchid"]
+    mosaic_id = resp["searchid"]
 
-    response = app.get(f"/mosaic/{cql2_id}/info")
+    response = app.get(f"/mosaic/{mosaic_id}/info")
     assert response.status_code == 200
     resp = response.json()
     assert resp["search"]
@@ -508,7 +509,7 @@ def test_query_with_metadata(app):
         "maxzoom": 2,
     }
 
-    response = app.get(f"/mosaic/{cql2_id}/tilejson.json?assets=cog")
+    response = app.get(f"/mosaic/{mosaic_id}/tilejson.json?assets=cog")
     assert response.status_code == 200
     resp = response.json()
     assert resp["minzoom"] == 1
@@ -527,28 +528,86 @@ def test_query_with_metadata(app):
             "defaults": {
                 "one_band": {
                     "assets": "cog",
-                    "asset_bidx": 1,
+                    "asset_bidx": "cog|1",
                 },
                 "three_bands": {
                     "assets": "cog",
-                    "asset_bidx": [1, 2, 3],
+                    "asset_bidx": "cog|1,2,3",
+                },
+                # missing `assets`
+                "bad_layer": {
+                    "asset_bidx": "cog|1,2,3",
                 },
             },
         },
     }
 
-    response = app.post("/mosaic/register", json=query)
+    with pytest.warns(UserWarning):
+        response = app.post("/mosaic/register", json=query)
     assert response.status_code == 200
     resp = response.json()
     assert resp["searchid"]
     assert (
         len(resp["links"]) == 6
     )  # info, tilejson, map, wmts tilejson for one_band, tilejson for three_bands
     link = resp["links"][-2]
+
+    mosaic_id_metadata = resp["searchid"]
+
     assert link["title"] == "TileJSON link for `one_band` layer."
-    assert "asset_bidx=1" in link["href"]
+    assert "asset_bidx=cog%7C1" in link["href"]
     assert "assets=cog" in link["href"]
 
+    # Test WMTS
+    # 1. missing assets and no metadata layers
+    response = app.get(f"/mosaic/{mosaic_id}/WMTSCapabilities.xml")
+    assert response.status_code == 400
+    assert (
+        response.json()["detail"]
+        == "assets must be defined either via expression or assets options."
+    )
+
+    # 2. assets and no metadata layers
+    response = app.get(
+        f"/mosaic/{mosaic_id}/WMTSCapabilities.xml", params={"assets": "cog"}
+    )
+    assert response.status_code == 200
+    assert response.headers["content-type"] == "application/xml"
+
+    with rasterio.open(io.BytesIO(response.content)) as src:
+        assert src.crs == "epsg:3857"
+        assert src.profile["driver"] == "WMTS"
+        assert not src.subdatasets
+
+    # 3. no assets and metadata layers
+    with pytest.warns(UserWarning):
+        response = app.get(f"/mosaic/{mosaic_id_metadata}/WMTSCapabilities.xml")
+    assert response.status_code == 200
+
+    assert response.headers["content-type"] == "application/xml"
+
+    with rasterio.open(io.BytesIO(response.content)) as src:
+        assert src.profile["driver"] == "WMTS"
+        assert len(src.subdatasets) == 2
+        assert src.subdatasets[0].endswith(",layer=one_band")
+        assert src.subdatasets[1].endswith(",layer=three_bands")
+
+    # 4. assets and metadata layers
+    with pytest.warns(UserWarning):
+        response = app.get(
+            f"/mosaic/{mosaic_id_metadata}/WMTSCapabilities.xml",
+            params={"assets": "cog"},
+        )
+    assert response.status_code == 200
+    assert response.headers["content-type"] == "application/xml"
+
+    with rasterio.open(io.BytesIO(response.content)) as src:
+        assert src.profile["driver"] == "WMTS"
+        assert len(src.subdatasets) == 3
+        assert src.subdatasets[0].endswith(",layer=one_band")
+        assert src.subdatasets[1].endswith(",layer=three_bands")
+        assert src.subdatasets[2].endswith(",layer=default")
+
 
 @patch("rio_tiler.io.rasterio.rasterio")
 def test_statistics(rio, app):