Skip to content

Introduce a custom StorageOptions Mapping #459

@ap--

Description

@ap--

Currently we rely on types.MappingProxyType to return storage_options as a read-only mapping under the .storage_options property. It would be a nice addition if we'd introduce a StorageOptions mapping class that would provide convenience methods for specific conversions:

  • .to_unchained()
    This style is supported by fsspec.open() where for a chained urlpath, every filesystem's options are provided under the corresponding protocol key. I.e.
    "zip::file://tmp/archive.zip", {"zip": {'some': 'option'}, "file": {'another': 'option'}}
  • .to_object_store()
    This could be used to translate to object_store storage_options from s3fs, az, and gcsfs storage options, and would be interesting for use with polars, and all the other rust tools.

We could also add a nicer __repr__ with a pretty formatted dict.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions