Skip to content

Commit

Permalink
refactor: Simplify data compression logic
Browse files Browse the repository at this point in the history
refactor: Simplify data compression logic

Remove duplicate file reading and streamline compression process

Co-authored-by: Daijiro Fukuda <[email protected]>
  • Loading branch information
ddukbg and daipom authored Nov 2, 2024
1 parent 5a4b255 commit 0759c78
Showing 1 changed file with 1 addition and 5 deletions.
6 changes: 1 addition & 5 deletions lib/fluent/plugin/out_s3.rb
Original file line number Diff line number Diff line change
Expand Up @@ -641,11 +641,7 @@ def content_type
end

def compress(chunk, tmp)
uncompressed_data = ''
chunk.open do |io|
uncompressed_data = io.read
end
compressed_data = Zstd.compress(uncompressed_data, level: @level)
compressed_data = Zstd.compress(chunk.read, level: @level)
tmp.write(compressed_data)
rescue => e
log.warn "zstd compression failed: #{e.message}"
Expand Down

0 comments on commit 0759c78

Please sign in to comment.