During the encoding process, memory usage increased by 5 times

[yanyouqi]

My filter has 100 million data, and it uses about 1GB of memory. During the encoding process, the memory increases to almost 7GB. What optimization can be done for encoding and decoding to reduce the intermediate memory consumption

[Ashutosh1406]

My filter has 100 million data, and it uses about 1GB of memory. During the encoding process, the memory increases to almost 7GB. What optimization can be done for encoding and decoding to reduce the intermediate memory consumption

Try binary Serialisation of data maybe something such as protobuf might help to encode and decode data faster.