Skip to content

Releases: apache/opendal

v0.49.1

20 Aug 07:19
@Xuanwo Xuanwo
v0.49.1
This tag was signed with the committer’s verified signature. The key has expired.
Xuanwo Xuanwo
GPG key ID: E95B7C7CB352F6C1
Expired
Verified
Learn about vigilant mode.
468fca7
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.49.1

Release List

Name Next
core 0.49.1
integrations/dav-server 0.0.8
integrations/fuse3 0.0.5
integrations/object_store 0.46.1
integrations/parquet 0.1.1
integrations/unftp-sbe 0.0.5
bin/oay 0.41.9
bin/ofs 0.0.10
bin/oli 0.41.9
bindings/c 0.44.11
bindings/cpp 0.45.9
bindings/dotnet 0.1.7
bindings/go 0.1.1
bindings/haskell 0.44.9
bindings/java 0.47.1
bindings/lua 0.1.7
bindings/nodejs 0.47.3
bindings/php 0.1.7
bindings/python 0.45.8
bindings/ruby 0.1.7

What's Changed

Added

  • feat(ovfs): add lookup and unit tests by @zjregee in #4997
  • feat(gcs): allow setting a token directly by @jdockerty in #4978
  • feat(integrations/cloudfilter): introduce behavior tests by @ho-229 in #4973
  • feat(integrations/spring): add spring project module by @shoothzj in #4988
  • feat(fs): expose the metadata for fs services by @Aitozi in #5005
  • feat(ovfs): add file creation and deletion by @zjregee in #5009

Fixed

  • fix(integrations/spring): correct parent artifactId by @shoothzj in #5007
  • fix(bindings/python): Make sure read until EOF by @Bicheka in #4995

Docs

  • docs: Fix version detect in website by @Xuanwo in #5003
  • docs: add branding, license and trademarks to integrations by @PsiACE in #5006
  • docs(integrations/cloudfilter): improve docs and examples by @ho-229 in #5010

CI

  • ci(bindings/python): Fix aws-lc-rs build on arm platforms by @Xuanwo in #5004

Chore

New Contributors

Full Changelog: v0.49.0...v0.49.1

Contributors

Xuanwo, Aitozi, and 7 other contributors
Loading
0 Join discussion

v0.49.0

13 Aug 09:51
@Xuanwo Xuanwo
v0.49.0
This tag was signed with the committer’s verified signature. The key has expired.
Xuanwo Xuanwo
GPG key ID: E95B7C7CB352F6C1
Expired
Verified
Learn about vigilant mode.
42c41ea
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.49.0

Release List

Name Next
core 0.49.0
integrations/dav-server 0.0.7
integrations/fuse3 0.0.4
integrations/object_store 0.46.0
integrations/parquet 0.1.0
integrations/unftp-sbe 0.0.4
bin/oay 0.41.8
bin/ofs 0.0.9
bin/oli 0.41.8
bindings/c 0.44.10
bindings/cpp 0.45.8
bindings/dotnet 0.1.6
bindings/go 0.1.0
bindings/haskell 0.44.8
bindings/java 0.47.0
bindings/lua 0.1.6
bindings/nodejs 0.47.2
bindings/php 0.1.6
bindings/python (*) 0.45.8
bindings/ruby 0.1.6
  • (*): We expected to release Python 0.45.8, but due to #5000, we were unable to. We will release it next time.

OpenDAL Core Upgrade to v0.49

Public API

Configurator now returns associated builder instead

Configurator used to return impl Builder, but now it returns associated builder type directly. This will allow users to use the builder in a more flexible way.

impl Configurator for MemoryConfig {
-    fn into_builder(self) -> impl Builder {
+    type Builder = MemoryBuilder;
+    fn into_builder(self) -> Self::Builder {
        MemoryBuilder { config: self }
    }
}

LoggingLayer now accepts LoggingInterceptor

LoggingLayer now accepts LoggingInterceptor trait instead of configuration. This change will allow users to customize the logging behavior more flexibly.

pub trait LoggingInterceptor: Debug + Clone + Send + Sync + Unpin + 'static {
    fn log(
        &self,
        info: &AccessorInfo,
        operation: Operation,
        context: &[(&str, &str)],
        message: &str,
        err: Option<&Error>,
    );
}

Users can now implement the log in the way they want.

OpenDAL Java Binding Upgrade to v0.47

Breaking change

artifactId of the opendal-java has changed from to opendal to align with the convention of entire OpenDAL project.

<dependencies>
<dependency>
  <groupId>org.apache.opendal</groupId>
-  <artifactId>opendal-java</artifactId>
+  <artifactId>opendal</artifactId>
  <version>${opendal.version}</version>
</dependency>
<dependency>
  <groupId>org.apache.opendal</groupId>
-  <artifactId>opendal-java</artifactId>
+  <artifactId>opendal</artifactId>
  <version>${opendal.version}</version>
  <classifier>${os.detected.classifier}</classifier>
</dependency>
</dependencies>

What's Changed

Added

  • feat(o): Add cargo-o layout by @Xuanwo in #4934
  • feat: impl put_multipart in object_store by @Rachelint in #4793
  • feat: introduce opendal AsyncWriter for parquet integrations by @WenyXu in #4958
  • feat(services/http): implement presigned request for backends without authorization by @NickCao in #4970
  • feat(bindings/python): strip the library for minimum file size by @NickCao in #4971
  • feat(gcs): allow unauthenticated requests by @jdockerty in #4965
  • feat: introduce opendal AsyncReader for parquet integrations by @WenyXu in #4972
  • feat(services/s3): add role_session_name in assume roles by @nerdroychan in #4981
  • feat: support root path for moka and mini-moka by @meteorgan in #4984
  • feat(ovfs): export VirtioFs struct by @zjregee in #4983
  • feat(core)!: implement an interceptor for the logging layer by @evenyag in #4961
  • feat(ovfs): support getattr and setattr by @zjregee in #4987

Changed

  • refactor(java)!: Rename artifacts id opendal-java to opendal by @Xuanwo in #4957
  • refactor(core)!: Return associated builder instead by @Xuanwo in #4968
  • refactor(raw): Merge all operations into one enum by @Xuanwo in #4977
  • refactor(core): Use kv based context to avoid allocations by @Xuanwo in #4986

Fixed

Docs

  • docs: nominate-committer add announcement template by @tisonkun in #4954

CI

Chore

New Contributors

Full Changelog: v0.48.0...v0.49.0

Contributors

meteorgan, Xuanwo, and 13 other contributors
Loading
0 Join discussion

v0.48.0

29 Jul 12:35
@Xuanwo Xuanwo
v0.48.0
This tag was signed with the committer’s verified signature. The key has expired.
Xuanwo Xuanwo
GPG key ID: E95B7C7CB352F6C1
Expired
Verified
Learn about vigilant mode.
aee7446
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.48.0

Release List

Name Version
core 0.48.0
integrations/cloudfilter 0.0.0
integrations/dav-server 0.0.6
integrations/fuse3 0.0.3
integrations/object_store 0.45.0
integrations/unftp-sbe 0.0.3
bin/oay 0.41.7
bin/ofs 0.0.8
bin/oli 0.41.7
bindings/c 0.44.9
bindings/cpp 0.45.7
bindings/dotnet 0.1.5
bindings/haskell 0.44.7
bindings/java 0.46.4
bindings/lua 0.1.5
bindings/nodejs 0.47.1
bindings/php 0.1.5
bindings/python 0.45.7
bindings/ruby 0.1.5

OpenDAL Core Upgrade to v0.48

Public API

Typo in customized_credential_load

Since v0.48, the customed_credential_load function has been renamed to customized_credential_load to fix the typo of customized.

- builder.customed_credential_load(v);
+ builder.customized_credential_load(v);

S3 service rename security_token to session_token

In 2014 Amazon switched from AWS_SECURITY_TOKEN to AWS_SESSION_TOKEN. To be consistent with the naming of AWS STS, we have renamed the security_token field to session_token in the S3 service.

- builder.security_token(v);
+ builder.session_token(v);

Operator from_iter and via_iter replaces from_map and via_map

Since v0.48, Operator's new APIs from_iter and via_iter methods have deprecated the from_map and via_map methods.

- Operator::from_map::<Fs>(map)?.finish();
+ Operator::from_iter::<Fs>(map)?.finish();

New API from_iter and via_iter should cover all use cases of from_map and via_map.

Service builder now takes ownership

Since v0.48, all service builder now takes ownership self instead of &mut self. This change will allow users to configure the service in a more flexible way.

- let mut builder = S3::default();
- builder.bucket("test");
- builder.root("/path/to/root");
+ let builder = S3::default().bucket("test").root("/path/to/root");
  let op = Operator::new(builder)?.finish();

Raw API

oio::Write::write will write the whole buffer

Starting from version 0.48, oio::Write::write now writes the entire buffer. This update aligns the API more closely with oio::Read::read and simplifies the implementation of concurrent writing.

  trait Write {
-     fn write(&mut self, bs: Buffer) -> impl Future<Output = Result<usize>>;
+     fn write(&mut self, bs: Buffer) -> impl Future<Output = Result<()>>;
  }

write will now return Result<()> instead of Result<usize>. The number of bytes written can be obtained from the buffer's length.

Access::metadata() will return Arc<AccessInfo>

Starting from version 0.48, Access::metadata() will return Arc<AccessInfo> instead of AccessInfo. This change is intended to improve performance and reduce memory usage.

  trait Access {
-     fn metadata(&self) -> AccessInfo;
+     fn metadata(&self) -> Arc<AccessInfo>;
  }

MinitraceLayer renamed to FastraceLayer

The MinitraceLayer has been renamed to FastraceLayer to respond to the transition from minitrace to fastrace.

- use opendal::layers::MinitraceLayer;
+ use opendal::layers::FastraceLayer;

Use Configurator to replace Builder::from_config

Since v0.48, the Builder::from_config and Builder::from_map method has been replaced by the Configurator trait. The Configurator trait provides a more flexible and extensible way to configure OpenDAL.

Service implementers should update their code to use the Configurator trait instead:

impl Configurator for MemoryConfig {
    fn into_builder(self) -> impl Builder {
        MemoryBuilder { config: self }
    }
}

impl Builder for MemoryBuilder {
    const SCHEME: Scheme = Scheme::Memory;
    type Config = MemoryConfig;

    fn build(self) -> Result<impl Access> {
        ...
    }
}

What's Changed

Added

  • feat(services/fs): Support fs config by @meteorgan in #4853
  • feat(services): init monoiofs by @NKID00 in #4855
  • feat(core/types): avoid a copy in Buffer::to_bytes() by cloning contiguous bytes by @LDeakin in #4858
  • feat(core): Add object versioning for OSS by @Lzzzzzt in #4870
  • feat: fs add concurrent write by @hoslo in #4817
  • feat(services/s3): Add object versioning for S3 by @Lzzzzzt in #4873
  • feat(integrations/cloudfilter): read only cloud filter by @ho-229 in #4856
  • feat(bindings/go): Add full native support from C to Go. by @yuchanns in #4886
  • feat(bindings/go): add benchmark. by @yuchanns in #4893
  • feat(core): support user defined metadata for oss by @meteorgan in #4881
  • feat(service/fastrace): rename minitrace to fastrace by @andylokandy in #4906
  • feat(prometheus-client): add metric label about root on using PrometheusClientLayer by @flaneur2020 in #4907
  • feat(services/monoiofs): monoio wrapper by @NKID00 in #4885
  • feat(layers/mime-guess): add a layer that can automatically set Content-Type based on the extension in the path. by @czy-29 in #4912
  • feat(core)!: Make config data object by @tisonkun in #4915
  • feat(core)!: from_map is now fallible by @tisonkun in #4917
  • ci(bindings/go): always test against the latest core by @yuchanns in #4913
  • feat(!): Allow users to build operator from config by @Xuanwo in #4919
  • feat: Add from_iter and via_iter for operator by @Xuanwo in #4921

Changed

  • refactor(services/s3)!: renamed security_token to session_token by @Zyyeric in #4875
  • refactor(core)!: Make oio::Write always write all given buffer by @Xuanwo in #4880
  • refactor(core)!: Return Arc<AccessInfo> for metadata by @Lzzzzzt in #4883
  • refactor(core!): Make service builder takes ownership by @Xuanwo in #4922
  • refactor(integrations/cloudfilter): implement Filter instead of SyncFilter by @ho-229 in #4920

Fixed

  • fix(services/s3): NoSuchBucket is a ConfigInvalid for OpenDAL by @tisonkun in #4895
  • fix: oss will not use the port by @Lzzzzzt in #4899

Docs

CI

  • ci: Add issue template and pr template for opendal by @Xuanwo in #4884
  • ci: Remove CI reviewer since it doesn't work by @Xuanwo in #4891

Chore

New Contributors

Full Changelog: v0.47.3...v0.48.0

Contributors

flaneur2020, jbampton, and 13 other contributors
Loading
0 Join discussion

v0.47.3

07 Jul 01:29
@Xuanwo Xuanwo
v0.47.3
This tag was signed with the committer’s verified signature. The key has expired.
Xuanwo Xuanwo
GPG key ID: E95B7C7CB352F6C1
Expired
Verified
Learn about vigilant mode.
f9df0fa
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.47.3

Release List

Name Version Next
core 0.47.2 0.47.3
integrations/dav-server 0.0.4 0.0.5
integrations/fuse3 0.0.1 0.0.2
integrations/object_store 0.44.2 0.44.3
integrations/unftp-sbe 0.0.1 0.0.2
bin/oay 0.41.5 0.41.6
bin/ofs 0.0.6 0.0.7
bin/oli 0.41.5 0.41.6
bindings/c 0.44.7 0.44.8
bindings/cpp 0.45.5 0.45.6
bindings/dotnet 0.1.3 0.1.4
bindings/haskell 0.44.5 0.44.6
bindings/java 0.46.2 0.46.3
bindings/lua 0.1.3 0.1.4
bindings/nodejs 0.47.0 0.47.0
bindings/php 0.1.3 0.1.4
bindings/python 0.45.5 0.45.6
bindings/ruby 0.1.3 0.1.4

What's Changed

Changed

  • refactor: Move ChunkedWrite logic into WriteContext by @Xuanwo in #4826
  • refactor(services/aliyun-drive): directly implement oio::Write. by @yuchanns in #4821

Fixed

  • fix(integration/object_store): Avoid calling API inside debug by @Xuanwo in #4846
  • fix(integration/object_store): Fix metakey requested is incomplete by @Xuanwo in #4844

Docs

  • docs(integration/unftp-sbe): Polish docs for unftp-sbe by @Xuanwo in #4838
  • docs(bin): Polish README for all bin by @Xuanwo in #4839

Chore

Full Changelog: v0.47.2...v0.47.3

Contributors

Xuanwo, TennyZhuang, and 2 other contributors
Loading
0 Join discussion

v0.47.2

03 Jul 05:08
@Xuanwo Xuanwo
v0.47.2
This tag was signed with the committer’s verified signature. The key has expired.
Xuanwo Xuanwo
GPG key ID: E95B7C7CB352F6C1
Expired
Verified
Learn about vigilant mode.
a58e802
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.47.2

Relelase List

Name Version Next
core 0.47.1 0.47.2
integrations/cloudfilter 0.0.0 0.0.0
integrations/dav-server 0.0.3 0.0.4
integrations/fuse3 0.0.0 0.0.1
integrations/object_store 0.44.1 0.44.2
integrations/unftp-sbe 0.0.0 0.0.1
integrations/virtiofs 0.0.0 0.0.0
bin/oay 0.41.4 0.41.5
bin/ofs 0.0.5 0.0.6
bin/oli 0.41.4 0.41.5
bindings/c 0.44.6 0.44.7
bindings/cpp 0.45.4 0.45.5
bindings/dotnet 0.1.2 0.1.3
bindings/go 0.0.0 0.0.0
bindings/haskell 0.44.4 0.44.5
bindings/java 0.46.1 0.46.2
bindings/lua 0.1.2 0.1.3
bindings/nodejs 0.46.2 0.47.0
bindings/ocaml 0.0.0 0.0.0
bindings/php 0.1.2 0.1.3
bindings/python 0.45.4 0.45.5
bindings/ruby 0.1.2 0.1.3
bindings/swift 0.0.0 0.0.0
bindings/zig 0.0.0 0.0.0
  • bindings/nodejs 0.47.0 release failed, we will get it fixed in next release.

Breaking changes

Binding Node.js Public API

Now, the append operation has been removed. You can use below code instead.

op.write("path/to/file", Buffer.from("hello world"), { append: true });

What's Changed

Added

Changed

  • refactor(nodejs)!: Remove append api by @bxb100 in #4796
  • refactor(core): Remove unused layer MadsimLayer by @zzzk1 in #4788

Fixed

  • fix(services/aliyun-drive): list dir without trailing slash by @yuchanns in #4766
  • fix(unftp-sbe): remove buffer for get by @George-Miao in #4775
  • fix(services/aliyun-drive): write op cannot overwrite existing files by @yuchanns in #4781
  • fix(core/services/onedrive): remove @odata.count for onedrive list op by @imWildCat in #4803
  • fix(core): Gcs's RangeWrite doesn't support concurrent write by @Xuanwo in #4806
  • fix(tests/behavior): skip test of write_with_overwrite for ghac by @yuchanns in #4823
  • fix(docs): some typos in website and nodejs binding docs by @suyanhanx in #4814
  • fix(core/aliyun_drive): Fix write_multi_max_size might overflow by @Xuanwo in #4830

Docs

CI

Chore

New Contributors

  • @zzzk1 made their first contribution in #4788

Full Changelog: v0.47.1...v0.47.2

Contributors

imWildCat, Xuanwo, and 10 other contributors
Loading
0 Join discussion

v0.47.1

21 Jun 11:00
@Xuanwo Xuanwo
v0.47.1
This tag was signed with the committer’s verified signature. The key has expired.
Xuanwo Xuanwo
GPG key ID: E95B7C7CB352F6C1
Expired
Verified
Learn about vigilant mode.
c98e81f
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.47.1

What's Changed

Added

  • feat(core): sets default chunk_size and sends buffer > chunk_size directly by @evenyag in #4710
  • feat(services): add optional access_token for AliyunDrive by @yuchanns in #4740
  • feat(unftp-sbe): Add integration for unftp-sbe by @George-Miao in #4753

Changed

  • refactor(ofs): Split fuse3 impl into fuse3_opendal by @Xuanwo in #4721
  • refactor(ovfs): Split ovfs impl into virtiofs_opendal by @zjregee in #4723
  • refactor(*): tiny refactor to the Error type by @waynexia in #4737
  • refactor(aliyun-drive): rewrite writer part by @yuchanns in #4744
  • refactor(object_store): Polish implementation details of object_store by @Xuanwo in #4749
  • refactor(dav-server): Polish dav-server integration details by @Xuanwo in #4751
  • refactor(core): Remove unused size for RangeWrite. by @reswqa in #4755

Fixed

  • fix(s3): parse MultipartUploadResponse to check error in body by @waynexia in #4735
  • fix(services/aliyun-drive): unable to list / by @yuchanns in #4754

Docs

CI

  • ci: verify build website on site content changes by @tisonkun in #4712
  • ci: Fix cert for redis and add docs for key maintenance by @Xuanwo in #4718
  • ci(nodejs): Disable services-all on windows by @Xuanwo in #4762

Chore

New Contributors

Full Changelog: v0.47.0...v0.47.1

Contributors

Xuanwo, evenyag, and 8 other contributors
Loading
0 Join discussion

v0.47.0

10 Jun 15:03
@tisonkun tisonkun
v0.47.0
This tag was signed with the committer’s verified signature.
tisonkun tison
GPG key ID: CD8564CB6A21E526
Verified
Learn about vigilant mode.
f2aa333
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.47.0

Core - Upgrade to v0.47

Public API

Reader into_xxx APIs

Since v0.47, Reader's into_xxx APIs requires async and returns Result instead.

- let r = op.reader("test.txt").await?.into_futures_async_read(1024..2048);
+ let r = op.reader("test.txt").await?.into_futures_async_read(1024..2048).await?;

Affected API includes:

  • Reader::into_futures_async_read
  • Reader::into_bytes_stream
  • BlockingReader::into_std_read
  • BlockingReader::into_bytes_iterator

Raw API

Bring Streaming Read Back

As explained in core: Bring Streaming Read Back, we do need read streaming back for better performance and low memory usage.

So our oio::Read changed back to streaming read instead:

trait Read {
-  async fn read(&self, offset: u64, size: usize) -> Result<Buffer>;
+  async fn read(&mut self) -> Result<Buffer>;
}

All services and layers should be updated to meet this change.

Java Binding - Upgrade to v0.46

Breaking change

PR-4641 renames async Operator to AsyncOperator and BlockingOperator to Operator.

New features

PR-4626 implements OperatorInputStream and OperatorOutputStream which implements Java's core IO abstractions InputStream and OutputStream. Users can now read/write bytes streamlined without loading/preparing the bytes fully in memory.

What's Changed

Added

  • feat(core/types): change oio::BlockingReader to Arc by @hoslo in #4577
  • fix: format_object_meta should not require metakeys that don't exist by @rebasedming in #4582
  • feat: add checksums to MultiPartComplete by @JWackerbauer in #4580
  • feat(doc): update object_store_opendal README by @hanxuanliang in #4606
  • feat(services/aliyun-drive): support AliyunDrive by @yuchanns in #4585
  • feat(bindings/python): Update type annotations by @3ok in #4630
  • feat: implement OperatorInputStream and OperatorOutputStream by @tisonkun in #4626
  • feat(bench): add buffer benchmark by @zjregee in #4603
  • feat: Add Executor struct and Execute trait by @Xuanwo in #4648
  • feat: Add executor in OpXxx and Operator by @Xuanwo in #4649
  • feat: Implement and refactor concurrent tasks for multipart write by @Xuanwo in #4653
  • feat(core/types): blocking remove_all for object storage based services by @TennyZhuang in #4665
  • feat(core): Streaming reading while chunk is not set by @Xuanwo in #4658
  • feat(core): Add more context in error context by @Xuanwo in #4673
  • feat: init ovfs by @zjregee in #4652
  • feat: Implement retry for streaming based read by @Xuanwo in #4683
  • feat(core): Implement TimeoutLayer for concurrent tasks by @Xuanwo in #4688
  • feat(core): Add reader size check in complete reader by @Xuanwo in #4690
  • feat(core): Azblob supports azure workload identity by @Xuanwo in #4705

Changed

  • refactor(core): Align naming for AccessorDyn by @morristai in #4574
  • refactor(core): core doesn't expose invalid input error anymore by @Xuanwo in #4632
  • refactor(core): Return unexpected error while content incomplete happen by @Xuanwo in #4633
  • refactor(core): Change Read's behavior to ensure it reads the exact size of data by @Xuanwo in #4634
  • refactor(bin/ofs): Fuse API by @ho-229 in #4637
  • refactor(binding/java)!: rename blocking and async operator by @tisonkun in #4641
  • refactor(core): Use concurrent tasks to refactor block write by @Xuanwo in #4692
  • refactor(core): Migrate RangeWrite to ConcurrentTasks by @Xuanwo in #4696

Fixed

  • fix(devcontainer/post_create.sh): change pnpm@stable to pnpm@latest by @GG2002 in #4584
  • fix(bin/ofs): privileged mount crashes when external umount by @ho-229 in #4586
  • fix(bin/ofs): ofs read only mount by @ho-229 in #4602
  • fix(raw): Allow retrying request while decoding response failed by @Xuanwo in #4612
  • fix(core): return None if metadata unavailable by @NKID00 in #4613
  • fix(bindings/python): Use abi3 and increase MSPV to 3.11 by @Xuanwo in #4623
  • fix: Fetch the content length while end_bound is unknown by @Xuanwo in #4631
  • fix: ofs write behavior by @ho-229 in #4617
  • fix(core/types): remove_all not work under object-store backend by @TennyZhuang in #4659
  • fix(ofs): Close file during flush by @Xuanwo in #4680
  • fix(core): RetryLayer could panic when other threads raises panic by @Xuanwo in #4685
  • fix(core/prometheus): Fix metrics from prometheus not correct for reader by @Xuanwo in #4691
  • fix(core/oio): Make ConcurrentTasks cancel safe by only pop after ready by @Xuanwo in #4707

Docs

CI

  • ci: upgrade typos to 1.21.0 and ignore changelog by @hezhizhen in #4601
  • ci: Disable jfrog webdav tests for it's keeping failed by @Xuanwo in #4607
  • ci: use official typos github action by @shoothzj in #4635
  • build(deps): upgrade crc32c to 0.6.6 for nightly toolchain by @tisonkun in #4650

Chore

New Contributors

Full Changelog: v0.46.0...v0.47.0

Contributors

jokester, Xuanwo, and 21 other contributors
Loading

v0.46.0

06 May 01:26
@tisonkun tisonkun
v0.46.0
This tag was signed with the committer’s verified signature.
tisonkun tison
GPG key ID: CD8564CB6A21E526
Verified
Learn about vigilant mode.
84586e5
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.46.0

Upgrade to v0.46

Public API

MSRV Changed to 1.75

Since 0.46, OpenDAL requires Rust 1.75.0 or later to use features like RPITIT and AFIT.

Services Feature Flag

Starting with version 0.46, OpenDAL only includes the memory service by default to prevent compiling unnecessary service code. To use other services, please activate their respective feature flags.

Additionally, we have removed all reqwest-related feature flags:

  • Users must now directly use reqwest's feature flags for options like rustls, native-tls, etc.
  • The rustls feature is no longer enabled by default; it must be activated manually.
  • OpenDAL no longer offers the trust-dns option; users should configure the client builder directly.

Range Based Read

Since v0.46, OpenDAL transformed it's Read IO trait to range based instead of stateful poll based IO. This change will make the IO more efficient, easier for concurrency and ready for completion based IO.

opendal::Reader now have APIs like:

let r = op.reader("test.txt").await?;
let buf = r.read(1024..2048).await?;

Buffer Based IO

Since version 0.46, OpenDAL features a native Buffer struct that supports both contiguous and non-contiguous buffers. This update enhances IO efficiency by minimizing unnecessary byte copying and enabling vectored IO.

OpenDAL's Reader will return Buffer and Writer will accept Buffer as input. Users who have implemented their own IO traits should update their code to use the new Buffer struct.

let r = op.reader("test.txt").await?;
// read returns `Buffer`
let buf: Buffer = r.read(1024..2048).await?;

let w = op.writer("test.txt").await?;

// Buffer can be created from continues bytes.
w.write("hello, world").await?;
// Buffer can also be created from non-continues bytes.
w.write(vec![Bytes::from("hello,"), Bytes::from("world!")]).await?;

// Make sure file has been written completely.
w.close().await?;

To enhance usability, we've integrated bridges into bytes::Buf and bytes::BufMut, allowing users to directly interact with the bytes API.

let r = op.reader("test.txt").await?;
let mut bs = vec![];
// read_into accepts bytes::BufMut
let buf: Buffer = r.read_into(&mut bs, 1024..2048).await?;

let w = op.writer("test.txt").await?;

// write_from accepts bytes::Buf
w.write_from("hello, world".as_bytes()).await?;

// Make sure file has been written completely.
w.close().await?;

Bridge API

OpenDAL's Reader and Writer previously implemented APIs such as AsyncRead and AsyncWrite directly. This design was not user-friendly, as it could lead to unexpected costs that users were unaware of in advance.

Since v0.46, OpenDAL provides bridge APIs for Reader and Writer instead.

let r = op.reader("test.txt").await?;

// Convert into futures AsyncRead + AsyncSeek.
let reader = r.into_futures_async_read(1024..2048);
// Convert into futures bytes stream.
let stream = r.into_bytes_stream(1024..2048);

let w = op.writer("test.txt").await?;

// Convert into futures AsyncWrite
let writer = w.into_futures_async_write();
// Convert into futures bytes sink;
let sink = w.into_bytes_sink();

Raw API

Async in IO trait

Since version 0.46, OpenDAL has adopted Rust's native async_in_trait for our core IO traits, including oio::Read, oio::Write, and oio::List.

This update eliminates the need for manually written, poll-based state machines and simplifies the codebase. Consequently, OpenDAL now requires Rust version 1.75.0 or later.

Users who have implemented their own IO traits should update their code to use the new async trait syntax.

What's Changed

Added

  • feat(services/github): add github contents support by @hoslo in #4281
  • feat: Allow selecting webpki for reqwest by @arlyon in #4303
  • feat(services/swift): add support for storage_url configuration in swift service by @zjregee in #4302
  • feat(services/swift): add ceph test setup for swift by @zjregee in #4307
  • docs(website): add local content search based on lunr plugin by @m1911star in #4348
  • feat(services/sled): add SledConfig by @yufan022 in #4351
  • feat : Implementing config for part of services by @AnuRage-git in #4344
  • feat(bindings/java): explicit async runtime by @tisonkun in #4376
  • feat(services/surrealdb): support surrealdb service by @yufan022 in #4375
  • feat(bindings/java): avoid double dispose NativeObject by @tisonkun in #4377
  • feat : Implement config for services/foundationdb by @AnuRage-git in #4355
  • feat: add ofs ctrl-c exit unmount hook by @oowl in #4393
  • feat: Implement RFC-4382 Range Based Read by @Xuanwo in #4381
  • feat(ofs): rename2 lseek copy_file_range getattr API support by @oowl in #4395
  • feat(services/github): make access_token optional by @hoslo in #4404
  • feat(core/oio): Add readable buf by @Xuanwo in #4407
  • feat(ofs): add freebsd OS support by @oowl in #4403
  • feat(core/raw/oio): Add Writable Buf by @Xuanwo in #4410
  • feat(bin/ofs): Add behavior test for ofs by @ho-229 in #4373
  • feat(core/raw/buf): Reduce one allocation by Arc::from_iter by @Xuanwo in #4440
  • feat: ?Send async trait for HttpBackend when the target is wasm32 by @waynexia in #4444
  • feat: add HttpClient::with() constructor by @waynexia in #4447
  • feat: Move Buffer as public API by @Xuanwo in #4450
  • feat: Optimize buffer implementation and add stream support by @Xuanwo in #4458
  • feat(core): Implement FromIterator for Buffer by @Xuanwo in #4459
  • feat(services/ftp): Support multiple write by @xxxuuu in #4425
  • feat(raw/oio): block write change to buffer by @hoslo in #4466
  • feat(core): Implement read and read_into for Reader by @Xuanwo in #4467
  • feat(core): Implement into_stream for Reader by @Xuanwo in #4473
  • feat(core): Tune buffer operations based on benchmark results by @Xuanwo in #4468
  • feat(raw/oio): Use Buffer as cache in RangeWrite by @reswqa in #4476
  • feat(raw/oio): Use Buffer as cache in OneshotWrite by @reswqa in #4477
  • feat: add list/copy/rename for dropbox by @zjregee in #4424
  • feat(core): Implement write and write_from for Writer by @zjregee in #4482
  • feat(core): Add auto ranged read and concurrent read support by @Xuanwo in #4486
  • feat(core): Implement fetch for Reader by @Xuanwo in #4488
  • feat(core): Allow concurrent reading on bytes stream by @Xuanwo in #4499
  • feat: provide send-wrapper to contidionally implement Send for operators by @waynexia in #4443
  • feat(bin/ofs): privileged mount by @ho-229 in #4507
  • feat(services/compfs): init compfs by @George-Miao in #4519
  • feat(raw/oio): Add PooledBuf to allow reuse buffer by @Xuanwo in #4522
  • feat(services/fs): Add PooledBuf in fs to allow reusing memory by @Xuanwo in #4525
  • feat(core): add access methods for Buffer by @George-Miao in #4530
  • feat(core): implement IoBuf for Buffer by @George-Miao in #4532
  • feat(services/compfs): compio runtime and compfs structure by @George-Miao in #4534
  • feat(core): change Result to default error by @George-Miao in #4535
  • feat(services/github): support list recursive by @hoslo in #4423
  • feat: Add crc32c checksums to S3 Service by @JWackerbauer in #4533
  • feat: Add into_bytes_sink for Writer by @Xuanwo in #4541

Changed

  • refactor(core/raw): Migrate oio::Read to async in trait by @Xuanwo in #4336
  • refactor(core/raw): Align oio::BlockingRead API with oio::Read by @Xuanwo in #4349
  • refactor(core/oio): Migrate oio::List to async fn in trait by @Xuanwo in #4352
  • refactor(core/raw): Migrate oio::Write from WriteBuf to Bytes by @Xuanwo in https://github...
Read more

Contributors

jbampton, mobiusklein, and 26 other contributors
Loading

v0.45.1

02 Mar 13:38
@Xuanwo Xuanwo
v0.45.1
This tag was signed with the committer’s verified signature. The key has expired.
Xuanwo Xuanwo
GPG key ID: A61687076CEAD424
Expired
Verified
Learn about vigilant mode.
5079125
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.45.1

Pacakages

Name Version
core 0.45.1
bin/oay 0.41.1
bin/oli 0.41.1
bin/ofs 0.0.2
bindings/python 0.45.1
bindings/nodejs 0.45.1
bindings/c 0.44.3
bindings/zig 0.0.0
bindings/dotnet 0.1.1
bindings/haskell 0.44.3
bindings/java 0.45.1
bindings/lua 0.1.1
bindings/ruby 0.1.1
bindings/swift 0.0.0
bindings/ocaml 0.0.0
bindings/php 0.1.1
bindings/cpp 0.44.3
bindings/go 0.0.0
integrations/object_store 0.43.0
integrations/dav-server 0.0.1

What's Changed

Added

  • feat(services/vercel_blob): support vercel blob by @hoslo in #4103
  • feat(bindings/python): add ruff as linter by @asukaminato0721 in #4135
  • feat(services/hdfs-native): Add capabilities for hdfs-native service by @jihuayu in #4174
  • feat(services/sqlite): Add list capability supported for sqlite by @jihuayu in #4180
  • feat(services/azblob): support multi write for azblob by @wcy-fdu in #4181
  • feat(release): Implement releasing OpenDAL components seperately by @Xuanwo in #4196
  • feat: object store adapter based on v0.9 by @waynexia in #4233
  • feat(bin/ofs): implement fuse for linux by @ho-229 in #4179
  • feat(services/memcached): change to binary protocal by @hoslo in #4252
  • feat(services/memcached): setup auth test for memcached by @hoslo in #4259
  • feat(services/yandex_disk): setup test for yandex disk by @hoslo in #4264
  • feat: add ci support for ceph_rados by @ZhengLin-Li in #4191
  • feat: Implement Config for part of services by @Xuanwo in #4277
  • feat: add jfrog test setup for webdav by @zjregee in #4265

Changed

  • refactor(bindings/python): simplify async writer aexit by @suyanhanx in #4128
  • refactor(service/d1): Add D1Config by @jihuayu in #4129
  • refactor: Rewrite webdav to improve code quality by @Xuanwo in #4280

Fixed

  • fix: Azdls returns 403 while continuation contains = by @Xuanwo in #4105
  • fix(bindings/python): missed to call close for the file internally by @zzl221000 in #4122
  • fix(bindings/python): sync writer exit close raise error by @suyanhanx in #4127
  • fix(services/chainsafe): fix 423 http status by @hoslo in #4148
  • fix(services/webdav): Add possibility to answer without response if file isn't exist by @AJIOB in #4170
  • fix(services/webdav): Recreate root directory if need by @AJIOB in #4173
  • fix(services/webdav): remove base_dir component by @hoslo in #4231
  • fix(core): Poll TimeoutLayer::sleep once to make sure timer registered by @Xuanwo in #4230
  • fix(services/webdav): Fix endpoint suffix not handled by @Xuanwo in #4257
  • fix(services/webdav): Fix possible error with value loosing from config by @AJIOB in #4262

Docs

CI

  • ci: Use old version of seafile mc instead by @Xuanwo in #4107
  • ci: Refactor workflows layout by @Xuanwo in #4139
  • ci: Migrate hdfs default setup by @Xuanwo in #4140
  • ci: Refactor check.sh into check.py to get ready for multi components release by @Xuanwo in #4159
  • ci: Add test case for hdfs over gcs bucket by @ArmandoZ in #4145
  • ci: Add hdfs test case for s3 by @ArmandoZ in #4184
  • ci: Add hdfs test case for azurite by @ArmandoZ in #4185
  • ci: Add support for releasing all rust packages by @Xuanwo in #4200
  • ci: Fix dependabot not update by @Xuanwo in #4202
  • ci: reduce the open pull request limits to 1 by @jbampton in #4225
  • ci: Remove version suffix from package versions by @Xuanwo in #4254
  • ci: Fix fuzz test for minio s3 name change by @Xuanwo in #4266
  • ci: Mark python 3.13 is not supported by @Xuanwo in #4269
  • ci: Disable yandex disk test for too slow by @Xuanwo in #4274
  • ci: Split python CI into release and checks by @Xuanwo in #4275
  • ci(release): Make sure LICENSE and NOTICE files are included by @Xuanwo in #4283
  • ci(release): Refactor and merge the check.py into verify.py by @Xuanwo in #4284

Chore

Read more

Contributors

jayvdb, jbampton, and 15 other contributors
Loading
0 Join discussion

v0.45.0

05 Feb 15:55
@morristai morristai
v0.45.0
This tag was signed with the committer’s verified signature.
morristai Morris
GPG key ID: DDBFEDDFD8121197
Verified
Learn about vigilant mode.
7190c07
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.45.0

Upgrade to v0.45

Core

Public API

BlockingLayer is not enabled by default

To further enhance the optionality of tokio, we have introduced a new feature called layers-blocking. The default usage of the blocking layer has been disabled. To utilize the BlockingLayer, please enable the layers-blocking feature.

TimeoutLayer deprecated with_speed

The with_speed API has been deprecated. Please use with_io_timeout instead.

Raw API

No raw API changes.

What's Changed

Added

  • feat(ofs): introduce ofs execute bin by @oowl in #4033
  • feat: add a missing news by @WenyXu in #4056
  • feat(services/koofr): set test for koofr by @suyanhanx in #4050
  • feat(layers/dtrace): Support User Statically-Defined Tracing(aka USDT) on Linux by @Zheaoli in #4053
  • feat(website): add missing news and organize the onboarding guide by @morristai in #4072
  • feat(services/chainsafe): setup test for chainsafe by @hoslo in #4089
  • docs: Add apache iceberg-rust as user by @liurenjie1024 in #4100

Changed

  • refactor!: avoid hard dep to tokio rt by @tisonkun in #4061
  • refactor: Implement IntoFuture for operator futures to remove an alloc by @Xuanwo in #4098

Fixed

  • fix(examples/cpp): display the results to standard output. by @SYaoJun in #4040
  • fix(service/icloud):Missing 'X-APPLE-WEBAUTH-USER cookie' and URL initialized failed by @bokket in #4029
  • fix: Implement timeout layer correctly by using timeout by @Xuanwo in #4059
  • fix(koofr): create_dir when exist by @hoslo in #4062
  • fix(seafile): test_list_dir_with_metakey by @hoslo in #4063
  • fix: list path recursive should not return path itself by @youngsofun in #4067
  • fix: Upgrade suppaftp to address build failure by @Xuanwo in #4091

Docs

  • docs: Remove not needed actions in release guide by @Xuanwo in #4037
  • docs: fix spelling errors in README.md by @SYaoJun in #4039
  • docs: New PMC member Liuqing Yue by @Xuanwo in #4047
  • docs: New Committer Yang Shuai by @Xuanwo in #4054
  • docs(services/sftp): add more explanation for endpoint config by @silver-ymz in #4055
  • docs: Add upgrade docs for core by @Xuanwo in #4095
  • docs: fix reference in rustdocs of FuturePresignWrite by @qrilka in #4097
  • docs: Add docs of options for xxx_with APIs by @Xuanwo in #4099

CI

  • ci(services/s3): Use minio/minio image instead by @Xuanwo in #4070
  • ci: Fix CI after moving out of workspacs by @Xuanwo in #4081

Chore

  • chore: Delete bindings/ruby/cucumber.yml by @tisonkun in #4030
  • chore(website): Bump download link for 0.44.2 release by @Zheaoli in #4034
  • chore(website): Update the release tips by @Zheaoli in #4036
  • chore: add doap file by @tisonkun in #4038
  • chore(website): Add extra artifacts check process in release document by @Zheaoli in #4041
  • chore(bindings/dotnet): update cargo.lock and set up ci by @suyanhanx in #4084
  • chore(bindings/dotnet): build os detect by @suyanhanx in #4085
  • chore(bindings/ocaml): pinning OCaml binding opendal version for release by @Ranxy in #4086
  • chore: Specify opendal version for haskell binding by @Xuanwo in #4093
  • chore: Bump to version 0.45.0 to start release process by @morristai in #4088

New Contributors

Full Changelog: v0.44.2...v0.45.0

Contributors

qrilka, liurenjie1024, and 14 other contributors
Loading