Bump pyo3-object_store to 0.1.0 (developmentseed#365)

Author: kylebarron

Cargo.lock

@@ -28,7 +28,7 @@ The simplest, highest-throughput [^1] Python interface to [S3][s3], [GCS][gcs],
 - Optionally return list results in [Apache Arrow](https://arrow.apache.org/) format, which is faster and more memory-efficient than materializing Python `dict`s.
 - Zero-copy data exchange between Rust and Python via the [buffer protocol](https://jakevdp.github.io/blog/2014/05/05/introduction-to-the-python-buffer-protocol/).
 
-<!-- For Rust developers looking to add object_store support to their Python packages, refer to pyo3-object_store. -->
+For Rust developers looking to add `object_store` support to their own Python packages, refer to [`pyo3-object_store`](https://docs.rs/pyo3-object_store/latest/pyo3_object_store/).
 
 [^1]: Benchmarking is ongoing, but preliminary results indicate roughly [9x higher throughput than fsspec](https://github.com/geospatial-jeff/pyasyncio-benchmark/blob/fe8f290cb3282dcc3bc96cae06ed5f90ad326eff/test_results/cog_header_results.csv) and [2.8x higher throughput than aioboto3](https://github.com/geospatial-jeff/pyasyncio-benchmark/blob/40e67509a248c5102a6b1608bcb9773295691213/test_results/20250218_results/ec2_m5/aggregated_results.csv) for many concurrent, small, get requests from an async context.
 

README.md

@@ -0,0 +1,5 @@
+# Changelog
+
+## [0.1.0] - 2025-03-14
+
+- Initial release.

pyo3-object_store/CHANGELOG.md

@@ -1,6 +1,6 @@
 [package]
 name = "pyo3-object_store"
-version = "0.1.0-beta.4"
+version = "0.1.0"
 authors = ["Kyle Barron <[email protected]>"]
 edition = "2021"
 description = "object_store integration for pyo3."

pyo3-object_store/Cargo.toml

@@ -2,37 +2,62 @@
 
 Integration between [`object_store`](https://docs.rs/object_store) and [`pyo3`](https://github.com/PyO3/pyo3).
 
-This provides Python builder classes so that Python users can easily create `Arc<dyn ObjectStore>` instances, which can then be used in pure-Rust code.
+This provides Python builder classes so that Python users can easily create [`Arc<dyn ObjectStore>`][object_store::ObjectStore] instances, which can then be used in pure-Rust code.
 
 ## Usage
 
 1. Register the builders.
 
-    ```rs
-    #[pymodule]
-    fn python_module(py: Python, m: &Bound<PyModule>) -> PyResult<()> {
-        pyo3_object_store::register_store_module(py, m, "python_module")?;
-        pyo3_object_store::register_exceptions_module(py, m, "python_module")?;
-    }
-    ```
+   ```rs
+   #[pymodule]
+   fn python_module(py: Python, m: &Bound<PyModule>) -> PyResult<()> {
+       pyo3_object_store::register_store_module(py, m, "python_module", "store")?;
+       pyo3_object_store::register_exceptions_module(py, m, "python_module", "exceptions")?;
+   }
+   ```
 
-    This exports the underlying Python classes from your own Rust-Python library.
+   This exports the underlying Python classes from your own Rust-Python library.
 
-2. Accept `PyObjectStore` as a parameter in your function exported to Python. Its `into_inner` method gives you an `Arc<dyn ObjectStore>`.
+   Refer to [`register_store_module`] and [`register_exceptions_module`] for more information.
 
-    ```rs
-    #[pyfunction]
-    pub fn use_object_store(store: PyObjectStore) {
-        let store: Arc<dyn ObjectStore> = store.into_inner();
-    }
-    ```
+2. Accept [`PyObjectStore`] as a parameter in your function exported to Python. Its [`into_dyn`][PyObjectStore::into_dyn] method (or `Into` impl) gives you an [`Arc<dyn ObjectStore>`][object_store::ObjectStore].
+
+   ```rs
+   #[pyfunction]
+   pub fn use_object_store(store: PyObjectStore) {
+       let store: Arc<dyn ObjectStore> = store.into_dyn();
+   }
+   ```
+
+   You can also accept [`AnyObjectStore`] as a parameter, which wraps [`PyObjectStore`] and [`PyExternalObjectStore`]. This allows you to seamlessly recreate `ObjectStore` instances that users pass in from other Python libraries (like [`obstore`][obstore]) that themselves export `pyo3-object_store` builders.
+
+   Note however that due to lack of [ABI stability](#abi-stability), `ObjectStore` instances will be **recreated**, and so there will be no connection pooling across the external store.
 
 ## Example
 
-The `obstore` Python library gives a full real-world example of using `pyo3-object_store`. It
+The [`obstore`][obstore] Python library gives a full real-world example of using `pyo3-object_store`, exporting a Python API that mimics the Rust [`ObjectStore`][object_store::ObjectStore] API.
+
+[obstore]: https://developmentseed.org/obstore/latest/
 
 ## ABI stability
 
+It's [not currently possible](https://github.com/PyO3/pyo3/issues/1444) to share a `#[pyclass]` across multiple Python libraries, except in special cases where the underlying data has a stable ABI.
+
+As `object_store` does not currently have a stable ABI, we can't share `PyObjectStore` instances across multiple separately-compiled Python libraries.
+
+We have two ways to get around this:
+
+- Export your own Python classes so that users can construct `ObjectStore` instances that were compiled _with your library_. See [`register_store_module`].
+- Accept [`AnyObjectStore`] or [`PyExternalObjectStore`] as a parameter, which allows for seamlessly **reconstructing** stores from an external Python library, like [`obstore`][obstore]. This has some overhead and removes any possibility of connection pooling across the two instances.
+
 Note about not being able to use these across Python packages. It has to be used with the exported classes from your own library.
 
-## Type hints
+## Python Type hints
+
+We don't yet have a _great_ solution here for reusing the store builder type hints in your own library. Type hints are shipped with the cargo dependency. Or, you can use a submodule on the `obstore` repo. See [`async-tiff` for an example](https://github.com/developmentseed/async-tiff/blob/35eaf116d9b1ab31232a1e23298b3102d2879e9c/python/python/async_tiff/store).
+
+## Version compatibility
+
+| pyo3-object_store | pyo3 | object_store |
+| ----------------- | ---- | ------------ |
+| 0.1.x             | 0.23 | 0.12         |