Skip to content

Add supports for Byte-Range Fetches when downloading from S3 #2981

@gael-ft

Description

@gael-ft

Short description

Add supports for Byte-Range Fetches when downloading an object from AWS S3.

It means doing like AWS "crt" based clients / TransferManager.

Details

Given:

  • bucket
  • S3 key
  • rangeLength (should default to 8MB)
  • parallelism

We should be able to download parallelism ranges in parallel.
Then pushing downstream them in order, requesting requesting a new range when first is complete.

Have my own implementation, I'll submit a PR

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions