extensions cannot render error pages without static_paths configuration

[minrk]

Description

Attempts to serve jupyter-server's default error pages from an ExtensionHandlerMixin handler fail with:

Exception: This extension doesn't have any static paths listed. Check that the extension's `static_paths` trait is set.

if the extension doesn't have static_paths configured. If static_paths is configured and doesn't include serverapp's static_paths, it renders the template successfully, but fails to find any static files:

This is because ExtensionHandlerMixin specifies that static_paths is per-extension, but static_url is used when rendering error page templates which come from base JupyterHandler and shouldn't be affected by the extension's configuration (at least unless overridden explicitly).

As it is now, essentially ExtensionHandlerMixin must always either:

1. guarantee it can never error, or
2. override write_error to not use error templates (e.g. APIHandler), or
3. ensure static_paths is set and includes serverapp.static_paths

This is currently coming up in jupyter_server_terminals when accessing a terminal that has stopped, reported in https://discourse.jupyter.org/t/this-extension-doesnt-have-any-static-paths-listed/26524 (the real topic there is what's causing the error, but it revealed the bug in error handling). Looking at the code, I think ExtensionHandlerMixin shouldn't be used in jupyter_server_terminals, but I don't think it should be broken if it is used.

I suspect the fix is to special-case the error templates to ensure JupyterHandler's static_url is called, and not the mixin's.

Reproduce

Create a test extension that raises an error:

from tornado import web
from jupyter_server.base.handlers import JupyterHandler
from jupyter_server.extension.handler import ExtensionHandlerMixin
from jupyter_server.extension.application import ExtensionApp

class TestHandler(ExtensionHandlerMixin, JupyterHandler):
    @web.authenticated
    def get(self):
        # should show default 404 page
        raise web.HTTPError(404)

class TestExtension(ExtensionApp):
    name = "test"
    default_url = "/test"
    load_other_extensions = False
    
    def initialize_handlers(self):
        self.handlers.extend([
            ("/test", TestHandler),
        ])

def _jupyter_server_extension_points():
    return [{"module": __name__, "app": TestExtension}]

if __name__ == "__main__":
    TestExtension.launch_instance()

Launch it with:

JUPYTER_NO_CONFIG=1 python test_ext.py --debug

default URL hits the 404 above, resulting in:

[E 2024-06-27 09:38:08.862 ServerApp] Uncaught exception in write_error
    Traceback (most recent call last):
      File "/Users/minrk/.virtualenvs/test-server/lib/python3.10/site-packages/tornado/web.py", line 1298, in send_error
        self.write_error(status_code, **kwargs)
      File "/Users/minrk/.virtualenvs/test-server/lib/python3.10/site-packages/jupyter_server/base/handlers.py", line 739, in write_error
        html = self.render_template("%s.html" % status_code, **ns)
      File "/Users/minrk/.virtualenvs/test-server/lib/python3.10/site-packages/jupyter_server/base/handlers.py", line 667, in render_template
        return template.render(**ns)
      File "/Users/minrk/.virtualenvs/test-server/lib/python3.10/site-packages/jinja2/environment.py", line 1304, in render
        self.environment.handle_exception()
      File "/Users/minrk/.virtualenvs/test-server/lib/python3.10/site-packages/jinja2/environment.py", line 939, in handle_exception
        raise rewrite_traceback_stack(source=source)
      File "/Users/minrk/.virtualenvs/test-server/lib/python3.10/site-packages/jupyter_server/templates/404.html", line 1, in top-level template code
        {% extends "error.html" %}
      File "/Users/minrk/.virtualenvs/test-server/lib/python3.10/site-packages/jupyter_server/templates/error.html", line 1, in top-level template code
        {% extends "page.html" %}
      File "/Users/minrk/.virtualenvs/test-server/lib/python3.10/site-packages/jupyter_server/templates/page.html", line 9, in top-level template code
        {% block favicon %}<link id="favicon" rel="shortcut icon" type="image/x-icon" href="{{ static_url("favicon.ico") }}">
      File "/Users/minrk/.virtualenvs/test-server/lib/python3.10/site-packages/jupyter_server/templates/page.html", line 9, in block 'favicon'
        {% block favicon %}<link id="favicon" rel="shortcut icon" type="image/x-icon" href="{{ static_url("favicon.ico") }}">
      File "/Users/minrk/.virtualenvs/test-server/lib/python3.10/site-packages/jupyter_server/extension/handler.py", line 118, in static_url
        raise Exception(msg) from None
    Exception: This extension doesn't have any static paths listed. Check that the extension's `static_paths` trait is set.

Expected behavior

default error templates render for extension pages without needing to specify unused static_paths config.

Context

• Operating System and version: macOS 14
• Browser and version: none
• Jupyter Server version: 2.14.1

Troubleshoot Output

$PATH:
	/Users/minrk/.virtualenvs/test-server/bin
	/Users/minrk/.local/bin
	/Users/minrk/conda/bin
	/opt/homebrew/bin
	/opt/homebrew/sbin
	/usr/local/bin
	/System/Cryptexes/App/usr/bin
	/usr/sbin
	/sbin
	/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin
	/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin
	/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin
sys.path:

/Users/minrk/.virtualenvs/test-server/bin

/Users/minrk/conda/lib/python310.zip

/Users/minrk/conda/lib/python3.10

/Users/minrk/conda/lib/python3.10/lib-dynload

/Users/minrk/.virtualenvs/test-server/lib/python3.10/site-packages
sys.executable:

/Users/minrk/.virtualenvs/test-server/bin/python
sys.version:

7.10.13 | packaged by conda-forge | (main, Dec 23 2023, 15:35:25) [Clang 16.0.6 ]
platform.platform():

macOS-14.5-arm64-arm-64bit
which -a jupyter:

/Users/minrk/.virtualenvs/test-server/bin/jupyter

/Users/minrk/conda/bin/jupyter
pip list:

Package                   Version

------------------------- --------------

anyio                     4.4.0

argon2-cffi               23.1.0

argon2-cffi-bindings      21.2.0

arrow                     1.3.0

attrs                     23.2.0

beautifulsoup4            4.12.3

bleach                    6.1.0

cffi                      1.16.0

defusedxml                0.7.1

exceptiongroup            1.2.1

fastjsonschema            2.20.0

fqdn                      1.5.1

idna                      3.7

isoduration               20.11.0

Jinja2                    3.1.4

jsonpointer               3.0.0

jsonschema                4.22.0

jsonschema-specifications 2023.12.1

jupyter_client            8.6.2

jupyter_core              5.7.2

jupyter-events            0.10.0

jupyter_server            2.14.1

jupyter_server_terminals  0.5.3

jupyterlab_pygments       0.3.0

MarkupSafe                2.1.5

mistune                   3.0.2

nbclient                  0.10.0

nbconvert                 7.16.4

nbformat                  5.10.4

overrides                 7.7.0

packaging                 24.1

pandocfilters             1.5.1

pip                       24.0

platformdirs              4.2.2

prometheus_client         0.20.0

ptyprocess                0.7.0

pycparser                 2.22

Pygments                  2.18.0

python-dateutil           2.9.0.post0

python-json-logger        2.0.7

PyYAML                    6.0.1

pyzmq                     26.0.3

referencing               0.35.1

rfc3339-validator         0.1.4

rfc3986-validator         0.1.1

rpds-py                   0.18.1

Send2Trash                1.8.3

setuptools                70.0.0

six                       1.16.0

sniffio                   1.3.1

soupsieve                 2.5

terminado                 0.18.1

tinycss2                  1.3.0

tornado                   6.4.1

traitlets                 5.14.3

types-python-dateutil     2.9.0.20240316

typing_extensions         4.12.2

uri-template              1.3.0

webcolors                 24.6.0

webencodings              0.5.1

websocket-client          1.8.0

wheel                     0.43.0

Command Line Output

> JUPYTER_NO_CONFIG=1 python test_ext.py --debug
[D 2024-06-27 09:38:07.376 ServerApp] No files in /Users/minrk/.ipython/nbextensions
[D 2024-06-27 09:38:07.376 ServerApp] Searching ['/var/folders/qr/3vxfnp1x2t1fw55dr288mphc0000gn/T/jupyter-clean-cfg-yjc3ufei'] for config files
[D 2024-06-27 09:38:07.376 ServerApp] Looking for jupyter_config in /var/folders/qr/3vxfnp1x2t1fw55dr288mphc0000gn/T/jupyter-clean-cfg-yjc3ufei
[D 2024-06-27 09:38:07.377 ServerApp] Looking for jupyter_server_config in /var/folders/qr/3vxfnp1x2t1fw55dr288mphc0000gn/T/jupyter-clean-cfg-yjc3ufei
[D 2024-06-27 09:38:07.381 ServerApp] Extension package __main__ took 0.0000s to import
[I 2024-06-27 09:38:07.382 ServerApp] __main__ | extension was successfully linked.
[D 2024-06-27 09:38:07.383 ServerApp] Raising open file limit: soft 256->4096; hard 9223372036854775807->9223372036854775807
[I 2024-06-27 09:38:07.400 ServerApp] __main__ | extension was successfully loaded.
[I 2024-06-27 09:38:07.403 ServerApp] test is running without loading other extensions.
[I 2024-06-27 09:38:07.403 ServerApp] Serving notebooks from local directory: /Users/minrk/dev/temp
[I 2024-06-27 09:38:07.403 ServerApp] Jupyter Server 2.14.1 is running at:
[I 2024-06-27 09:38:07.403 ServerApp] http://localhost:8888/test?token=1d1e587a06fbf199ad6b280d5c2700c11b183b80ea904b1c
[I 2024-06-27 09:38:07.403 ServerApp]     http://127.0.0.1:8888/test?token=1d1e587a06fbf199ad6b280d5c2700c11b183b80ea904b1c
[I 2024-06-27 09:38:07.403 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 2024-06-27 09:38:07.410 ServerApp]
To access the server, open this file in a browser:
    file:///Users/minrk/Library/Jupyter/runtime/jpserver-38849-open.html
Or copy and paste one of these URLs:
    http://localhost:8888/test?token=1d1e587a06fbf199ad6b280d5c2700c11b183b80ea904b1c
    http://127.0.0.1:8888/test?token=1d1e587a06fbf199ad6b280d5c2700c11b183b80ea904b1c

[D 2024-06-27 09:38:08.851 ServerApp] Accepting token-authenticated request from ::1

[D 2024-06-27 09:38:08.851 TestExtension] Generating new user for token-authenticated request: 3dd6c5d501ce466d80f1bda5c9872276

[D 2024-06-27 09:38:08.852 TestExtension] Using contents: services/contents

[E 2024-06-27 09:38:08.862 ServerApp] Uncaught exception in write_error

Traceback (most recent call last):

File "/Users/minrk/.virtualenvs/test-server/lib/python3.10/site-packages/tornado/web.py", line 1298, in send_error

self.write_error(status_code, **kwargs)

File "/Users/minrk/.virtualenvs/test-server/lib/python3.10/site-packages/jupyter_server/base/handlers.py", line 739, in write_error

html = self.render_template("%s.html" % status_code, **ns)

File "/Users/minrk/.virtualenvs/test-server/lib/python3.10/site-packages/jupyter_server/base/handlers.py", line 667, in render_template

return template.render(**ns)

File "/Users/minrk/.virtualenvs/test-server/lib/python3.10/site-packages/jinja2/environment.py", line 1304, in render

self.environment.handle_exception()

File "/Users/minrk/.virtualenvs/test-server/lib/python3.10/site-packages/jinja2/environment.py", line 939, in handle_exception

raise rewrite_traceback_stack(source=source)

File "/Users/minrk/.virtualenvs/test-server/lib/python3.10/site-packages/jupyter_server/templates/404.html", line 1, in top-level template code

{% extends "error.html" %}

File "/Users/minrk/.virtualenvs/test-server/lib/python3.10/site-packages/jupyter_server/templates/error.html", line 1, in top-level template code

{% extends "page.html" %}

File "/Users/minrk/.virtualenvs/test-server/lib/python3.10/site-packages/jupyter_server/templates/page.html", line 9, in top-level template code

{% block favicon %}<link id="favicon" rel="shortcut icon" type="image/x-icon" href="{{ static_url("favicon.ico") }}">

File "/Users/minrk/.virtualenvs/test-server/lib/python3.10/site-packages/jupyter_server/templates/page.html", line 9, in block 'favicon'

{% block favicon %}<link id="favicon" rel="shortcut icon" type="image/x-icon" href="{{ static_url("favicon.ico") }}">

File "/Users/minrk/.virtualenvs/test-server/lib/python3.10/site-packages/jupyter_server/extension/handler.py", line 118, in static_url

raise Exception(msg) from None

Exception: This extension doesn't have any static paths listed. Check that the extension's static_paths trait is set.

[W 2024-06-27 09:38:08.866 TestExtension] 404 GET /test?token=[secret] (3dd6c5d501ce466d80f1bda5c9872276@::1) 16.35ms referer=None

[D 2024-06-27 09:38:08.945 ServerApp] Path favicon.ico served from /Users/minrk/.virtualenvs/test-server/lib/python3.10/site-packages/jupyter_server/static/favicon.ico

[D 2024-06-27 09:38:08.946 ServerApp] 200 GET /favicon.ico (3dd6c5d501ce466d80f1bda5c9872276@::1) 2.34ms

[minrk]

Looking a bit more, since static_url and templates are fundamentally tied together, and due to the presence of error pages, I think perhaps ExtensionHandlerJinjaMixin should be merged into ExtensionHandlerMixin, and if the fallback path of super().get_template is taken, static_url and static_path should come from super() not from the mixin.

Perhaps the best fix is to apply static_url differently - i.e. instead of being an instance method on the Handler objects, place it into the Environment globals. Because static_url really is a property of the. I'm not sure of the backward-compatibility implications of switching to doing it that way, though.

[invisibleroads]

Seeing this error even in a vanilla jupyterlab install when starting the terminal extension

[E 2024-10-30 17:57:51.925 ServerApp] Uncaught exception in write_error
    Traceback (most recent call last):
      File "/.virtualenvs/crosscompute/lib64/python3.12/site-packages/tornado/web.py", line 1298, in send_error
        self.write_error(status_code, **kwargs)
      File "/.virtualenvs/crosscompute/lib64/python3.12/site-packages/jupyter_server/base/handlers.py", line 739, in write_error
        html = self.render_template("%s.html" % status_code, **ns)

      ...

      File "/.virtualenvs/crosscompute/lib64/python3.12/site-packages/jupyter_server/extension/handler.py", line 118, in static_url
        raise Exception(msg) from None
    Exception: This extension doesn't have any static paths listed. Check that the extension's `static_paths` trait is set.