Failed to decompress zstd message

[slashmili]

I'm having problem consuming data which are compressed using zstd (#619):

10:57:51.651 [error] Consumer my-app-consumer terminate reason: {"failed to decompress: ZSTD_CONTENTSIZE_UNKNOWN",
 [
   {:kpro_compress, :iodata, 1,
    [
      file: ~c"/Users/user/my_app/deps/kafka_protocol/src/kpro_compress.erl",
      line: 112
    ]},
   {:kpro_batch, :do_decode, 2,
    [
      file: ~c"/Users/user/my_app/deps/kafka_protocol/src/kpro_batch.erl",
      line: 199
    ]},
   {:kpro_batch, :decode, 2,
    [
      file: ~c"/Users/user/my_app/deps/kafka_protocol/src/kpro_batch.erl",
      line: 129
    ]},
   {:brod_utils, :parse_fetch_rsp, 1,
    [
      file: ~c"/Users/user/my_app/deps/brod/src/brod_utils.erl",
      line: 787
    ]},
   {:brod_utils, :parse_rsp, 1,
    [
      file: ~c"/Users/user/my_app/deps/brod/src/brod_utils.erl",
      line: 615
    ]},
   {:brod_consumer, :handle_fetch_response, 2,
    [
      file: ~c"/Users/user/my_app/deps/brod/src/brod_consumer.erl",
      line: 510
    ]},
   {:gen_server, :try_handle_info, 3, [file: ~c"gen_server.erl", line: 2173]},
   {:gen_server, :handle_msg, 6, [file: ~c"gen_server.erl", line: 2261]}
 ]}

I'm not the producer of this data so I can't give insight about that but if I consume the same topic using Java SDK, it works. I need to dig further to provide more details.

I'm planning to see if I use brod to produce data using zstd, can I consume it or not. Also planning to extract to read the raw binary data from this topic and try to decompress it manually.

I'm using

ezstd -> 1.2
brod -> 4.4
Erlang/OTP -> 27