v0.11.0

🚨 Breaking Changes

This release marks the end of support for Python 3.9 (and earlier), which has reached end-of-life status with the release of Python 3.13.

What's Changed

• Update project lockfiles and pre-commit setup by @nicholasjng in #287
• Bump oldest supported Python to 3.10 by @nicholasjng in #288
• [ci] Update CI workflow to be based on uv toolchain by @nicholasjng in #289
• Run pre-commit autoupdate again by @nicholasjng in #290
• Support fs.exists() on repo root by @AdrianoKF in #292

Full Changelog: v0.10.0...v0.11.0

v0.10.0

This release fixes a case in which users attempting to bulk-delete more than 1000 objects would get errors from the lakeFS server. From now on, object deletes are carried out in batches of 1000, which is the maximum that the server handles.

What's Changed

• Update lockfiles for CI and dev setups by @nicholasjng in #281
• Batch objects in Branch.delete_objects calls by @nicholasjng in #285
• Add rm test on more than 1000 files by @nicholasjng in #286

Full Changelog: v0.9.0...v0.10.0

v0.9.0

This release increases the minimum required fsspec version to fsspec>=2023.12.0 (December 2023). It does not contain changes in functionality.

What's Changed

• Bump minimum fsspec version to 2023.12.0 by @nicholasjng in #280
• Update pre-commit hooks, change lockfile to use uv pip compile by @nicholasjng in #278

Full Changelog: v0.8.0.post0...v0.9.0

v0.8.0.post0 - Documentation updates

This release adds example code showing how to access lakeFS repositories from the Hugging Face 🤗 Datasets library to the documentation.

What's Changed

• Docs: Demonstrate integration of Hugging Face Datasets by @AdrianoKF in #275
• Bump linter dependencies by @AdrianoKF in #276

Full Changelog: v0.8.0...v0.8.0.post0

v0.8.0

This release changes the behavior of LakeFSFileSystem.exists() when encountering a CommonPrefix, which is lakeFS' directory abstraction. Afterwards, the file system can detect existing directories/common prefixes on a branch, returning True for any existing directories.

What's Changed

• Update outdated CI actions by @AdrianoKF in #269
• Update pre-commit hooks by @AdrianoKF in #270
• Fixed broken URL in README.md by @leonpawelzik in #271
• Docs: Link to root-level modules in API docs by @AdrianoKF in #273
• fs.exists() should return True for common prefixes by @ozkatz in #274

New Contributors

• @leonpawelzik made their first contribution in #271
• @ozkatz made their first contribution in #274

Full Changelog: v0.7.1...v0.8.0

v0.7.1

What's Changed

• Update Python dependencies, change ruff options by @nicholasjng in #262
• Fix ls by @renesat in #260
• Log a warning for uncommitted changes on transaction exit by @maxmynter in #266
• Always show the project name in the header by @janwillemkl in #263
• Docs: Fix code highlighting in third-party libraries integration guide by @AdrianoKF in #268

New Contributors

• @renesat made their first contribution in #260

Full Changelog: v0.7.0...v0.7.1

v0.7.0

⭐ Highlights

Changes the lakeFS-spec transaction to operate on short-lived ephemeral branches (#258).

Transaction versioning operations are implicitly scoped to the given repository and transaction branch, so you do not need to manually specify these arguments anymore:

# will automatically merge into `main` upon successful completion of `tx`.
with fs.transaction("my-repo", "main") as tx:
    fs.put("my-file.txt", f"my-repo/{tx.branch.id}/my-file.txt")
    tx.commit(message="Add my file")

🚨 Breaking changes

The usage of LakeFSFileSystem.transaction was changed. It is now necessary to call it with a repository name to properly initialize the transaction.

What's Changed

• Bump lakeFS service container version to v1.7.0 by @nicholasjng in #249
• Replace fs._open() with proper fs.open() implementation by @nicholasjng in #250
• Add integration tests for third-party data science libraries by @nicholasjng in #251
• (docs): Add lakeFS-spec and lakeFS-SDK transaction differences by @maxmynter in #252
• Update project metadata by @AdrianoKF in #256
• Split lakefs-spec test job into multiple jobs by @nicholasjng in #261
• Introduce new transaction model based on ephemeral branches by @nicholasjng in #258

Full Changelog: v0.6.1...v0.7.0

v0.6.1

⭐ Highlights

This release marks the completion of the lakeFS SDK wrapper migration. All available functionality is now expressed in terms of abstractions from the new lakefs Python package.

As a particular consequence, the LakeFSFile was removed in favor of lakeFS' own file-like object APIs. For simple file interactions through the file system only (including transactions), this will not result in breaking changes.

What's Changed

• Slim down the bug report and feature request templates by @janwillemkl in #235
• Update dev requirements, fix lakefs v0.2.0 regression by @nicholasjng in #239
• Implement more APIs in terms of lakefs abstractions by @nicholasjng in #240
• Fix fs.rm() for the recursive case by @nicholasjng in #243
• Add test coverage for commits to repo information tests by @nicholasjng in #244
• Remove Placeholder from interface typing, use new ReferenceType hint by @nicholasjng in #245
• Add lakeFS package versions to pytest header by @nicholasjng in #246
• Update bug-report.yaml by @Maciej818 in #247
• Simplify revision parsing in the tutorial notebook by @nicholasjng in #248

Full Changelog: v0.6.0...v0.6.1

v0.6.0 - New lakeFS SDK

⭐ Highlights

• lakeFS-spec is now using the newly release lakefs high-level Python SDK under the hood (see #231)
• the project now has a logo in all the relevant places 🎨

🚨 Breaking Changes

• Loading lakectl.yaml configuration from non-default paths is no longer supported (in line with the lakefs Python SDK)

What's Changed

• Correctly set docs version number for release workflow by @AdrianoKF in #228
• Docs: Update project highlights by @janwillemkl in #229
• Several small test improvements by @nicholasjng in #230
• Migrate the package to use the new lakefs SDK wrapper library by @nicholasjng in #231
• Implement deletes using the lakeFS branch API by @maxmynter in #236
• Add custom lakeFS-spec logo/favicon by @AdrianoKF in #238

New Contributors

• @janwillemkl made their first contribution in #229

Full Changelog: v0.5.0...v0.6.0

v0.5.0

⭐ Highlights

The v0.5.0 release of lakeFS-spec simplifies working with deferred placeholder entities returned by versioning operations as part of a transaction. These placeholders now work as transparent proxy objects, meaning they expose the same interface as the class they are wrapping (no more Placeholder.unwrap() necessary! 🙌🏻 )

The user documentation has been completed, especially the remaining missing pieces of the API reference.

🚨 Breaking Changes

• LakeFSFileSystem.ls() no longer defaults to recursive listing, in line with other fsspec implementations. If you require recursive directory listings, you can pass the recursive=True argument to ls() or consider the find()/glob()/walk() operations.

What's Changed

• Allow Path like inputs in public LakeFSFileSystem methods by @maxmynter in #182
• Create API docstrings for the LakeFSFileSystem and LakeFSFile by @nicholasjng in #181
• Docs: Allow Jupytext notebooks for tutorials by @AdrianoKF in #183
• Add API docs for all remaining smaller modules by @nicholasjng in #184
• Record test coverage on fsspec package [ENG-196] by @AdrianoKF in #188
• Modify error translator to raise more informative errors by @nicholasjng in #190
• Idempotency of demo notebook by @maxmynter in #189
• Drop requirements.txt from package lock files by @nicholasjng in #194
• Docs: Fix typos in tx user guide by @AdrianoKF in #197
• Bump lakeFS version for testing environment to v1.3.0 by @AdrianoKF in #199
• Docs: Add call to action for early adopters to README and landing page [ENG-186] by @AdrianoKF in #196
• ls: Make non-recursive by default, correctly update dircache by @AdrianoKF in #198
• Introduce issue templates by @Maciej818 in #202
• Revert "Introduce issue templates" by @AdrianoKF in #204
• Improve test coverage on lakefs abstract base classes by @AdrianoKF in #201
• Introduce templates - eliminate yaml errrors by @Maciej818 in #205
• Validate with docstrings with pydoclint by @AdrianoKF in #203
• Add lakeFS server version guard to LakeFSFilesystem.touch() by @nicholasjng in #214
• Overload LakeFSFile.__del__ to pass statement by @nicholasjng in #219
• Remove bug ftr prefix issue templates by @Maciej818 in #217
• Add overloads for LakeFSFileSystem.ls() by @nicholasjng in #221
• Docs: Client-side caching for Open-Meteo API in tutorial notebook by @AdrianoKF in #220
• Run pre-commit autoupdate by @nicholasjng in #223
• Remove dunder methods from API docs by @nicholasjng in #222
• Create template pull-request by @Maciej818 in #206
• Correctly handle dircache for recursive ls calls by @AdrianoKF in #224
• Simplify transaction Placeholder class by @maxmynter in #225
• Fix transaction doc on Placeholder values by @nicholasjng in #226

New Contributors

• @Maciej818 made their first contribution in #202

Full Changelog: v0.4.0...v0.5.0