Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Zstd compression support to S3 plugin #439

Merged
merged 13 commits into from
Nov 6, 2024
Merged

Commits on Oct 18, 2024

  1. Bump actions/checkout from 3 to 4

    Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4.
    - [Release notes](https://github.com/actions/checkout/releases)
    - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
    - [Commits](actions/checkout@v3...v4)
    
    ---
    updated-dependencies:
    - dependency-name: actions/checkout
      dependency-type: direct:production
      update-type: version-update:semver-major
    ...
    
    Signed-off-by: dependabot[bot] <[email protected]>
    Signed-off-by: yongwoo.kim <[email protected]>
    dependabot[bot] authored and yongwoo.kim committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    ed12eed View commit details
    Browse the repository at this point in the history
  2. feat: Add Zstd compression support for S3 plugin

    Signed-off-by: yongwoo.kim <[email protected]>
    yongwoo.kim committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    796d6c9 View commit details
    Browse the repository at this point in the history

Commits on Oct 19, 2024

  1. Change zstd-ruby to runtime dependencies

    Signed-off-by: ddukbg <[email protected]>
    ddukbg committed Oct 19, 2024
    Configuration menu
    Copy the full SHA
    57ba441 View commit details
    Browse the repository at this point in the history

Commits on Oct 23, 2024

  1. Remove redundant bundler installation from GitHub Actions workflow

    Before:
    - name: Install dependencies
      run: gem install bundler rake
    
    After:
    - name: Install dependencies
      run: gem install rake
    
    Signed-off-by: ddukbg <[email protected]>
    ddukbg committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    6f84c65 View commit details
    Browse the repository at this point in the history

Commits on Oct 25, 2024

  1. Configuration menu
    Copy the full SHA
    7f6e51c View commit details
    Browse the repository at this point in the history
  2. Remove ZstdCompressor tests from test_in_s3.rb as per maintainer's co…

    …mments
    
    Moved ZstdCompressor tests from test_in_s3.rb to test_out_s3.rb as they relate to the out_s3 plugin.
    
    Signed-off-by: ddukbg <[email protected]>
    ddukbg committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    6aa84f4 View commit details
    Browse the repository at this point in the history
  3. Add ZstdCompressor test cases to test_out_s3.rb as per maintainer's c…

    …omments
    
    Added tests for ZstdCompressor to test_out_s3.rb following the maintainer's suggestions.
    
    Signed-off-by: ddukbg <[email protected]>
    ddukbg committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    dd3bafb View commit details
    Browse the repository at this point in the history
  4. Merge pull request #2 from ddukbg/fix/pr-comment

    Add ZstdCompressor to S3 Plugin and Fix Tests According to Maintainer's Feedback
    ddukbg authored Oct 25, 2024
    Configuration menu
    Copy the full SHA
    0ad8cdf View commit details
    Browse the repository at this point in the history

Commits on Nov 2, 2024

  1. refactor: Remove unnecessary whitespace

    Remove redundant spaces to improve code readability and consistency
    
    Co-authored-by: Daijiro Fukuda <[email protected]>
    Signed-off-by: ddukbg <[email protected]>
    
    refactor: Simplify data compression logic
    
    refactor: Simplify data compression logic
    
    Remove duplicate file reading and streamline compression process
    
    Co-authored-by: Daijiro Fukuda <[email protected]>
    Signed-off-by: ddukbg <[email protected]>
    ddukbg and daipom committed Nov 2, 2024
    Configuration menu
    Copy the full SHA
    7e28a32 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    904a043 View commit details
    Browse the repository at this point in the history

Commits on Nov 6, 2024

  1. test: Add tests for Zstd compression level configuration

    Add test cases to verify:
    - Default compression level (3)
    - Custom compression level setting
    
    Co-authored-by: Daijiro Fukuda <[email protected]>
    Signed-off-by: ddukbg <[email protected]>
    ddukbg and daipom committed Nov 6, 2024
    Configuration menu
    Copy the full SHA
    e31e889 View commit details
    Browse the repository at this point in the history
  2. fix: Improve ZSTD compression implementation - Remove unnecessary lin…

    …e ending and encoding handling - Fix compression logic using chunk.write_to method - Properly implement ZstdCompressor in separate file The previous implementation mistakenly included explicit line ending and encoding handling during troubleshooting. This has been removed as the proper implementation using chunk.write_to handles the data correctly without such manipulation. Signed-off-by: ddukbg <[email protected]>
    
    Signed-off-by: ddukbg <[email protected]>
    ddukbg committed Nov 6, 2024
    Configuration menu
    Copy the full SHA
    fa2a609 View commit details
    Browse the repository at this point in the history
  3. refactor: Simplify ZSTD compression implementation

    Use direct chunk.read method instead of intermediate StringIO.
    This simplification maintains the same functionality while
    making the code more straightforward and easier to understand.
    
    Co-authored-by: Daijiro Fukuda <[email protected]>
    Signed-off-by: ddukbg <[email protected]>
    ddukbg and daipom committed Nov 6, 2024
    Configuration menu
    Copy the full SHA
    08448a1 View commit details
    Browse the repository at this point in the history