Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Return BoxStream with 'static lifetime from ObjectStore::list #6619

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

kylebarron
Copy link
Contributor

@kylebarron kylebarron commented Oct 23, 2024

To preface: not expecting this to be merged imminently, as it may be a while before object-store's next breaking release, and I figure there may be a cleaner way to change this. But it was useful to learn a bit more about streams, and I'll probably use this patch in my Python-facing object-store bindings.

Which issue does this PR close?

Closes #6587.

Rationale for this change

Easier wrapping of ObjectStore::list in bindings, e.g. to Python, that don't allow lifetime references.

What changes are included in this PR?

Change the return type of ObjectStore::list and ObjectStore::list_with_offset to

BoxStream<'static, Result<ObjectMeta>>

Are there any user-facing changes?

Yes.

@github-actions github-actions bot added the object-store Object Store Interface label Oct 23, 2024
@kylebarron
Copy link
Contributor Author

kylebarron commented Oct 23, 2024

In developmentseed/obstore#35 I verified that this does enable streaming list in Python bindings:

Screen.Recording.2024-10-23.at.12.25.19.PM.mov

@tustvold tustvold added api-change Changes to the arrow API next-major-release the PR has API changes and it waiting on the next major version labels Oct 23, 2024
@alamb alamb requested a review from tustvold November 7, 2024 22:46
@alamb
Copy link
Contributor

alamb commented Nov 7, 2024

Looks like @tustvold had a conversation on #6587 so I defer to him. However, I had a quick look at the code and it looks pretty good to me. Thank you @kylebarron

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api-change Changes to the arrow API next-major-release the PR has API changes and it waiting on the next major version object-store Object Store Interface
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Use 'static lifetime in BoxFuture for all object-store APIs
3 participants