Skip to content

Commit

Permalink
fix: Enhance ZstdCompressor with proper binary handling
Browse files Browse the repository at this point in the history
Signed-off-by: ddukbg <[email protected]>
  • Loading branch information
ddukbg committed Nov 2, 2024
1 parent 8e3589d commit 85bccb5
Showing 1 changed file with 20 additions and 5 deletions.
25 changes: 20 additions & 5 deletions lib/fluent/plugin/out_s3.rb
Original file line number Diff line number Diff line change
Expand Up @@ -648,14 +648,29 @@ def content_type

def compress(chunk, tmp)
begin
data = chunk.open { |io| io.read }
compressed = Zstd.compress(data, level: @level)
tmp.write(compressed)
tmp.flush
tmp.close
data = ''
chunk.open { |io| data = io.read }

compressed = nil
begin
compressed = Zstd.compress(data, level: @level)
rescue => e
log.warn "Failed to compress with zstd: #{e.message}"
raise e
end

if compressed
tmp.binmode
tmp.write(compressed)
tmp.flush
else
raise "Failed to compress data with zstd"
end
rescue => e
log.warn "zstd compression failed: #{e.message}"
raise e
ensure
tmp.close rescue nil
end
end
end
Expand Down

0 comments on commit 85bccb5

Please sign in to comment.