Listing products with `dc.list_products()` produces many odc-geo GridSpec warnings in Datacube 1.9

[robbibt]

In Datacube 1.9, the dc.list_products() function products a large number of odc-geo GridSpec warnings. These make it difficult to use these functions to get a list of available Datcube products:

import datacube

dc = datacube.Datacube()
dc.list_products()

[omad]

Thanks Robbi. The PR Arianna has opened it's a good start, but I think there's a few things to do to fix it properly.

There's two changes in 1.9 that are related, but need to be addressed differently.

Developer API

We deprecated the internal GridSpec class and switched to odc-geo. This change is mostly only relevant developers using the ODC APIs, and can help avoid some weird gotchas.

We can use deprecat and python warnings to surface this information to developers that they need to make the switch to the new API.

Product specifications

As part of that, we also now expect Product grid specifications be to use tile_shape as an integer number of pixels. Instead of the old way with tile_size being in CRS units. This change is relevant to data managers and people writing or indexing products, since we now want product specifications to use the new field.

I haven't traced through the code yet, but I think these warning messages are because there are products in your database, and potentially in the specs in git that are using the old tile_size field.

There's a few approaches we can take here, and I don't have it all mapped out in my head.

These sorts of warnings still need to exist to let users know, but they are about the data being used, and not about anything wrong with the code. Which means they shouldn't be coming from the python warnings system since it's very much code and developer focused. Using logged warnings here is the correct approach.

However, they need to be logged from an appropriate place to provide actionable information, which in this case would be something like "Warning: Product x_y_z has it's grid defined using tile_size which as of ODC 1.9. Please get your ODC administrator to update the product definition. See https://odc-docs-link for more information.".

Other potential code changes

• If there is still internal code using the deprecated class it needs to be updated. It's easy to convert between the tile size and tile shape representation (while logging a warning) when we load a product, and to then use the correct API.
• We should make sure we're alerting users any time we're loading an old style product specification, adding, etc.
• We should make sure we don't have old tests or examples lying around in the old style. (I know that there are in Explorer for example)
• We could provide example docs for an admin to audit and update product specs.

[robbibt]

Thanks @omad! I think warnings like these can definitely be useful - I just aren't sure that they should be surfaced in a top level, commonly used function like dc.list_products(). That function is one of the first things that get run by beginner users who just want to learn what products they can load. Those users don't really have the ability to influence how data was originally indexed, so the warnings end up creating a lot of distracting noise.

(we also have an ongoing issue where beginner users see red warnings and immediately assume they have broken something, so they tend to have a disproportionate impact on usability).

[pjonsson]

• We should make sure we don't have old tests or examples lying around in the old style. (I know that there are in Explorer for example)

The remaining deprecation warnings in Explorer are caused by accessing the grid_spec property. The accesses don't need the entire grid_spec, but they need some part of it and there isn't an API in core for getting those parts without triggering the deprecation warning as far as I know.

(I effectively copied the code from core into Explorer to eliminate those warnings in opendatacube/datacube-explorer#658, but that also eliminates the @cached_property decorator of grid_spec which I assume was put there for a good reason.)