:question: [Question]: Can we clean up the jupyterhub_config.py code

[mishaschwartz]

Questions

The code in jupyterhub_config.py.template is difficult to read and edit. Issues are:

• some customization are template variables and others are environment variables (no clear reason why except maybe historical?)
• inconsistent modification of spawner and authenticator attributes. For example, some modify attributes for c.Spawner or c.DockerSpawner or c.CustomSpawner (all work but its confusing why we pick one over the other)
• custom authenticator is defined in https://github.com/Ouranosinc/jupyterhub but the custom Spawner is defined in jupyterhub_config.py.template
• no linting, no tests

In the past this might have worked but this file has gotten complex enough that it makes more sense to treat it like a proper python project

Proposed Solution

Move the CustomSpawner implementation and most of the hardcoded settings to to https://github.com/Ouranosinc/jupyterhub.

Keep jupyterhub_config.py.template for customizations that can differ for different deployments as well as hardcoded values for the c.JupyterHub instance.

This will make this file easier to maintain and also put the code for both the CustomSpawner and MagpieAuthenticator in the same place so that they can be maintained together.

References

Environment

Information	Value
Server/Platform URL
Version Tag/Commit
Related issues/PR
Related components

[tlvu]

• some customization are template variables and others are environment variables (no clear reason why except maybe historical?)

Yes, that was historical organic grow reason. We started out as env var for single value customization, then moved to template variables when a block of customization was required to be added.

• inconsistent modification of spawner and authenticator attributes. For example, some modify attributes for c.Spawner or c.DockerSpawner or c.CustomSpawner (all work but its confusing why we pick one over the other)

No particular reasons, probably copy/paste from the examples. I don't see c.CustomSpawner ?

• custom authenticator is defined in https://github.com/Ouranosinc/jupyterhub but the custom Spawner is defined in jupyterhub_config.py.template

No particular reasons, the person who wrote the MagpieAuthenticator decided to put it there.

Originally https://github.com/Ouranosinc/jupyterhub was created so we can install the dockerspawner package into the jupyterhub image to avoid install on the fly via an entrypoint (like in components/canarie-api/entrypoint) that is not reproducible.

• no linting, no tests

True, because the changes were simple so far. But I agree if you want to add some tests and restructure the code, it'll be a good idea.

Move the CustomSpawner implementation and most of the hardcoded settings to to https://github.com/Ouranosinc/jupyterhub.

Keep jupyterhub_config.py.template for customizations that can differ for different deployments as well as hardcoded values for the c.JupyterHub instance.

This will make this file easier to maintain and also put the code for both the CustomSpawner and MagpieAuthenticator in the same place so that they can be maintained together.

How about the reverse? Move MapgieAuthenticator back to birdhouse-deploy? Having the code outside means for each change, we need to make a new release for the JupyterHub image before we can pull the change in. Having the code external to birdhouse-deploy also make it harder to experiment changes on a live Birdhouse test instance. For me, all Birdhouse customizations, for all components, should all be in the same place. That very tiny MagpieAuthenticator was an outlier.

Inside birdhouse-deploy, you can also structure the code the way you want and volume mount the additional files into the jupyterhub container. You don't have to make all the new changes inside the same jupyterhub_config.py.template file.

We already have pytest running in birdhouse-deploy, we can easily add additional tests to cover the JupyterHub customization.

[mishaschwartz]

Sure I'm happy to move the MagpieAuthenticator to this repo as well. That solves the same problem

[fmigneault]

I agree with @tlvu's last points about moving into birdhouse-deploy. I do not see how Ouranosinc/jupyterhub could be used in any other context than within birdhouse-deploy, since both the CustomDockerSpawner and MagpieAuthenticator rely on its specific multi-services setup. It would be clearer to centralize the changes needed on them within the same components.

Most of the c.DockerSpawner..../c.Spawner.... structures are indeed based on reference documentation, which makes it easier to figure out (and directly search) which parameter defined in the documentation they refer to. Since simply overriding c.JupyterHub.spawner_class = CustomDockerSpawner is enough to make them take effect without adjusting the c.DockerSpawner.... names, I personally think it is better to have them defined this way. Even though c.DockerSpawner inherits the c.Swpawner configs, the reference JupyterHub documentation also indicates examples using the c.Swpawner rather than c.DockerSwpawner base (eg: see code in https://jupyterhub-dockerspawner.readthedocs.io/en/latest/api/index.html#dockerspawner.DockerSpawner.pre_spawn_hook). I think it is easier to preserve the same conventions to facilitate lookup of the doc/config, which is already hard enough to interpret.

On CRIM's side and inside the CI, we only override using the following, with some inline python conditional logic to filter/align images with upstream latest references within the {...} block.

export ENABLE_JUPYTERHUB_MULTI_NOTEBOOKS="
c.DockerSpawner.image_whitelist = { ... }
"

As long as this override remains functional, I don't mind the other reshuffling of template/env-vars definitions if it can simplify the customization for other use cases.

[tlvu]

As long as this override remains functional, I don't mind the other reshuffling of template/env-vars definitions if it can simplify the customization for other use cases.

Good point. Please preserve backward compatibility (all the env vars and template expansion vars). Otherwise, you can move the code around as you see fit for better organization and ease of understanding.