Skip to content

Remove vendored zarr code #617

@TomNicholas

Description

@TomNicholas
Member

We are apparently still vendoring some code from zarr-python, even though a while ago we added an explicit dependency on it (in #482). That vendored code should have been removed then in favour of directly importing things.

Activity

TomNicholas

TomNicholas commented on Jun 17, 2025

@TomNicholas
MemberAuthor

This came up when I got this error trying to run tests locally

ImportError while loading conftest '/Users/tom/Documents/Work/Code/VirtualiZarr/conftest.py'.
conftest.py:16: in <module>
    from virtualizarr.manifests import ChunkManifest, ManifestArray
virtualizarr/__init__.py:3: in <module>
    from virtualizarr.accessor import (
virtualizarr/accessor.py:7: in <module>
    from virtualizarr.manifests import ManifestArray
virtualizarr/manifests/__init__.py:7: in <module>
    from virtualizarr.manifests.store import ManifestStore, ObjectStoreRegistry  # type: ignore # noqa
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
virtualizarr/manifests/store.py:20: in <module>
    from virtualizarr.vendor.zarr.core.metadata import dict_to_buffer
virtualizarr/vendor/zarr/core/metadata.py:6: in <module>
    from zarr.core.metadata.v3 import V3JsonEncoder
E   ImportError: cannot import name 'V3JsonEncoder' from 'zarr.core.metadata.v3' (/Users/tom/Documents/Work/Code/VirtualiZarr/.venv/lib/python3.13/site-packages/zarr/core/metadata/v3.py)

But actually this might be related to the zarr v3 dtypes refactor?

maxrjones

maxrjones commented on Jun 18, 2025

@maxrjones
Member

💯 agree on removing the vendored metadata code.

I think @norlandrhagen vendored https://github.com/zarr-developers/VirtualiZarr/blob/develop/virtualizarr/vendor/zarr/core/common.py to use a private function which we cannot rely on directly. So it might need to stay unless we find a better solution for https://discourse.pangeo.io/t/async-and-sync-interfaces/5190.

added this to the v2.0.1 milestone on Jul 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingzarr-pythonRelevant to zarr-python upstream

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @maxrjones@TomNicholas

        Issue actions

          Remove vendored zarr code · Issue #617 · zarr-developers/VirtualiZarr