Internal Server Error accessing `/stac/collections`.

[omad]

Our production ODC Explorer STAC API endpoint has stopped working. It returns an unhelpful "Internal Server Error has been logged" message.

Checking the application logs reveals a stack trace similar to the following.

Traceback (most recent call last):
  File "/Users/aye011/dev/datacube-explorer/.venv/lib/python3.13/site-packages/werkzeug/serving.py", line 370, in run_wsgi
    execute(self.server.app)
    ~~~~~~~^^^^^^^^^^^^^^^^^
  File "/Users/aye011/dev/datacube-explorer/.venv/lib/python3.13/site-packages/werkzeug/serving.py", line 331, in execute
    application_iter = app(environ, start_response)
  File "/Users/aye011/dev/datacube-explorer/.venv/lib/python3.13/site-packages/flask/app.py", line 1536, in __call__
    return self.wsgi_app(environ, start_response)
           ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/aye011/dev/datacube-explorer/.venv/lib/python3.13/site-packages/werkzeug/middleware/proxy_fix.py", line 183, in __call__
    return self.app(environ, start_response)
           ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/aye011/dev/datacube-explorer/.venv/lib/python3.13/site-packages/flask/app.py", line 1514, in wsgi_app
    response = self.handle_exception(e)
  File "/Users/aye011/dev/datacube-explorer/.venv/lib/python3.13/site-packages/flask_cors/extension.py", line 176, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
                                                ~^^^^^^^^^^^^^^^^^
  File "/Users/aye011/dev/datacube-explorer/.venv/lib/python3.13/site-packages/flask/app.py", line 1511, in wsgi_app
    response = self.full_dispatch_request()
  File "/Users/aye011/dev/datacube-explorer/.venv/lib/python3.13/site-packages/flask/app.py", line 919, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/Users/aye011/dev/datacube-explorer/.venv/lib/python3.13/site-packages/flask_cors/extension.py", line 176, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
                                                ~^^^^^^^^^^^^^^^^^
  File "/Users/aye011/dev/datacube-explorer/.venv/lib/python3.13/site-packages/flask/app.py", line 917, in full_dispatch_request
    rv = self.dispatch_request()
  File "/Users/aye011/dev/datacube-explorer/.venv/lib/python3.13/site-packages/flask/app.py", line 902, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/Users/aye011/dev/datacube-explorer/cubedash/_stac.py", line 1093, in collections
    _stac_collection(product.name).to_dict()
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
  File "/Users/aye011/dev/datacube-explorer/cubedash/_stac.py", line 877, in _stac_collection
    footprint = all_time_summary.footprint_wgs84
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'footprint_wgs84'

Upon firing up the debugger, I can finally see that there are a couple of products which have been summarised by Explorer (Have a record in cubedash.product), but don't have a period_type='all' record in cubedash.time_overview.

SELECT p.name
FROM cubedash.product AS p
WHERE p.id NOT IN (
    SELECT to_all.product_ref
    FROM cubedash.time_overview AS to_all
    WHERE to_all.period_type = 'all'
);

I don't know how this happened, but can only guess it's from an incomplete execution of cubedash-gen leaving the database in an inconsistent state, and cubedash server being unable to cope with it.

Affected Versions

I've confirmed this happens with at least 3.0.0 and 2.12.4

[omad]

I re-ran cubedash-get --refresh-stats --all and it found and generated the missing summaries for the 4 products that were missing the all period_type.

Now I'm receiving 504 Gateway Timeout errors when attempting to access the STAC Endpoint. I don't know if it's related or not.

I'll leave this open anyway, as

• The server code should be more resilient
• The server code should provide more useful logs
• If it is from a failing or aborted cubedash-gen run, it shouldn't leave the database in an unexpected state.