Allow customized Dask Cluster Name

[tracycarley]

Is that possible to reset the Dask Cluster Name so that users can potentially prefix their own cluster with their username? This seems useful in a multi-tenant environment. When using simple authentication, it provides a quick way for users to know which cluster they can reconnect to, esp in the situation when their client (notebook) is restarted but the Dask cluster is still in progress.
The code here indicates the cluster name is not settable. Not sure if there's any workaround with this?

[jabbera]

I just went through this. There doesn't seem to be an easy way to customize the cluster at all per user. I implemented my own backend and overrode start_cluster. We do other things here like set the run as uid and make user specific mounts. I'm a little surprised this wasn't stock functionality but it wasn't too difficult to implement.

class MyKubeBackend(KubeBackend):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        

    # we use this to customize the backend pods to the user spawning the cluster
    # this should be kept in sync with KubeBackend.start_cluster minus our customizations
    async def start_cluster(self, user, cluster_options):
        options, config = await self.process_cluster_options(user, cluster_options)

        obj = self.make_cluster_object(user.name, options, config)
        environment_variables = obj["spec"]["config"]["worker_extra_container_config"]["env"]

        # customize the user by populating the JUPYTERHUB_USER environment variable with their username
        user_env = next(filter(lambda x: x["name"] == "JUPYTERHUB_USER", environment_variables))
        user_env["value"] = user.name

        # customize the cluster name to always be: namespace.username
        cluster_name = f"{config.namespace}.{user.name}"
        obj["metadata"]["name"] = user.name

        self.log.info("Creating cluster %s for user %s", cluster_name, user.name)

        await self.custom_client.create_namespaced_custom_object("gateway.dask.org", self.crd_version, config.namespace, "daskclusters", obj)
        return cluster_name

[vdytyniak-exos]

@jabbera How to make server use custom MyKubeBackend? Thanks.

[jabbera]

I built a custom docker image.

[vdytyniak-exos]

Thanks. I think it is also possible to use extensions.

[tasansal]

Even though I think there are good reasons to not have this option, I think in some cases it may be useful to override that. I can make a PR to allow custom names. Would be good if we can some feedback from maintainers?

[consideRatio]

Thank you for asking @tasansal!!

I'm low on capacity and know other maintainers are as well, so for a non-essential feature that adds complexity and could be a breaking change during a transition period when changing naming patterns etc, I'd rather not see a PR opened.

I've wanted this myself at one point, but I've seen a lot of complexity arise from similar changes in other projects involving escaping names to meet restrictions on k8s resource names. I don't think its a complexity investment the projects maintenance capacity can afford.