AttributeError when displaying dask-gateway cluster object in a notebook in latest pangeo-notebook mage

[guillaumeeb]

This came up in pangeo-data/pangeo-eosc#22 (comment).

With a newly deployed Daskhub, using pangeo/pangeo-notebook:2022.09.21 Docker image, when creating and displaying a dask-gateway cluster using:

from dask_gateway import Gateway
gateway = Gateway()
cluster = gateway.new_cluster(worker_memory=4)
cluster.scale(6)
cluster

The following error is displayed:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
File /srv/conda/envs/notebook/lib/python3.9/site-packages/IPython/core/formatters.py:921, in IPythonDisplayFormatter.__call__(self, obj)
    919 method = get_real_method(obj, self.print_method)
    920 if method is not None:
--> 921     method()
    922     return True

File /srv/conda/envs/notebook/lib/python3.9/site-packages/dask_gateway/client.py:1225, in GatewayCluster._ipython_display_(self, **kwargs)
   1223 widget = self._widget()
   1224 if widget is not None:
-> 1225     return widget._ipython_display_(**kwargs)
   1226 else:
   1227     from IPython.display import display

AttributeError: 'VBox' object has no attribute '_ipython_display_'

This does not prevent using the cluster.

This was not present with pangeo/pangeo-notebook:2022.09.06.

I did not dig further, just opening this issue if someone has a guess.

[TomAugspurger]

That was fixed in dask/dask-gateway#609 (incompatibility with ipywidgets >= 0.8). There isn't a dask-gateway release with it yet, so you'll want to print(cluster) in the meantime.

[guillaumeeb]

Thanks @TomAugspurger! I could have search a bit more 😉.

I'll let you close this issue if you feel like it.

[yuvipanda]

is there a dask-gateway release now? we should bump here.

[scottyhq]

our pangeo-bot dispatcher to catch package updates needs a manual kickstart every 60 days :( not sure how to fix that but here is the issue pangeo-bot/dispatcher#2

[scottyhq]

pangeo/pangeo-notebook:2022.10.18 now has dask-gateway=2022.10.0 I did a quick test on the staging hub and things appear to work but print(cluster) doesn't return anything, just displaying the client connected to the cluster:

also, you have to put the full URL into the dask dashboard to get it to work (so something like https://staging.us-central1-b.gcp.pangeo.io/services/dask-gateway/clusters/staging.0b841d0d64b9473d9864b3c9a150aac9/status)

Not sure who is responsible for bumping the image version on the production 2i2c pangeo hub?

[rabernat]

Not sure who is responsible for bumping the image version on the production 2i2c pangeo hub?

I'm the hub owner, and it's not clear to me either! 🤦 @yuvipanda what do you suggest in terms of process here? I can't be just manually making PRs to https://github.com/2i2c-org/infrastructure every time a new image comes out. Related:

• Report Docker Image in Login Page or JupyterLab UI 2i2c-org/features#16
• Support choosing images in profile list for pangeo hubs 2i2c-org/infrastructure#1253

[yuvipanda]

@rabernat there's progress being made on 2i2c-org/infrastructure#1253 (2i2c-org/infrastructure#1825 was just merged today).

In the meantime, you don't actually need to make PRs to the repo each time, any jupyterhub admin can manually modify the image with the configurator: https://docs.2i2c.org/en/latest/admin/howto/configurator.html