v0.13.0

[romange]

Dragonfly v0.13.0

The first release of the year! Yey! 👯

This release contains a few exciting features.

• Active defrag: any long-running process with non-trivial allocation patterns suffers from excessive memory usage due to external fragmentation. This release uses an improved defragmentation algorithm after we released the initial version in the previous version.
• Experimental SSD-based data-tiering support. Dragonfly is the only in-memory store that provides data tiering that is faster than other memory-only stores. (Currently only for strings). Try it with flag --spill_file_prefix=<nvme_file_basepath>
• JSON - this release compliments the already existing JSON API with support for SCAN and TYPE working with JSON data types.
• Replication (still not for prod use) but can already be tried on simple single key commands like SET, SADD, ZADD, LPUSH etc.
• Writes and reads of large string values are optimized for x86 architectures with SIMD instructions.
• Numerous improvements to helm templates, plus docker health checks - thanks to @tamcore !

Huge thanks to all the contributors! ❤️

What's Changed

• feat(rdb save): Create compressor interface by @adiholden in feat(rdb save): Create compressor interface #538
• fix(helm): Chart.yaml - Add home, keywords, sources, kubeVersion by @arukiidou in fix(helm): Chart.yaml - Add home, keywords, sources, kubeVersion #542
• feat: add healtcheck for container by @tamcore in feat: add healtcheck for container #544
• Support lz4 compression by @adiholden in Support lz4 compression #545
• feat(server): json set type support by @boazsade in feat(server): json set type support #546
• chore: Update helio dependency by @romange in chore: Update helio dependency #553
• chore: Update helio dependency by @dranikpg in chore: Update helio dependency #555
• feat(server): Implement robust error & cancellation on replica by @dranikpg in feat(server): Implement robust error & cancellation on replica #531
• feat(ci test): add build and test on release mode by @adiholden in feat(ci test): add build and test on release mode #556
• added example of start script by @Tomato6966 in added example of start script #557
• Support offloading of blobs longer than 2KB by @romange in Support offloading of blobs longer than 2KB #559
• feat(server): active memory defrag test application by @boazsade in feat(server): active memory defrag test application #552
• feat(server): JSON family using JSON type by @boazsade in feat(server): JSON family using JSON type #561
• feat(server): New auto-journal types/read/write by @dranikpg in feat(server): New auto-journal types/read/write #560
• chore: Refactor RdbLoad by @romange in chore: Refactor RdbLoad #564
• chore: Adding a mpsc intrusive queue based on Vyukov's design by @romange in chore: Adding a mpsc intrusive queue based on Vyukov's design #562
• feat: introduce simd algorithm for bitpacking by @romange in feat: introduce simd algorithm for bitpacking #568
• feat: implement ascii_unpack using SIMD instructions by @romange in feat: implement ascii_unpack using SIMD instructions #573
• bug(rdb load): Rdb loader tasks stop running on failure Rdb loader tasks continue running on failure #567 by @adiholden in bug(rdb load): Rdb loader tasks stop running on failure #567 #576
• chore: Improve the implementation of simd based packing by @romange in chore: Improve the implementation of simd based packing #577
• fix: choose an appropriate SIMD implementation for aarch64 by @romange in fix: choose an appropriate SIMD implementation for aarch64 #579
• fix: handle corner case when comitted memory is negative by @romange in fix: handle corner case when comitted memory is negative #570
• feat(server): Use new journal format by @dranikpg in feat(server): Use new journal format #563
• fix(chart): probe indentation by @tamcore in fix(chart): probe indentation #590
• feat(server): Redesign and simplify tiered storage module. by @romange in feat(server): Redesign and simplify tiered storage module. #589
• use dense_set for hashmap family by @romange in use dense_set for hashmap family #593
• feat(server): adding version monitoring task by @boazsade in feat(server): adding version monitoring task #586
• feat(replica): Support FlushDB command for replication Support FlushDB command for replication #580 by @adiholden in feat(replica): Support FlushDB command for replication #580 #591
• fix(server): defrag unit test fix by @boazsade in fix(server): defrag unit test fix #602
• chore(server): upgrade mimalloc for v2.0.7 by @boazsade in chore(server): upgrade mimalloc for v2.0.7 #597
• feat: build and publish weekly alpha/development container image by @tamcore in feat: build and publish weekly alpha/development container image #594
• life of a transaction by @romange in life of a transaction #278
• feat(server): track hits and misses stats. by @ashotland in feat(server): track hits and misses stats. #608
• feat(server): Improved cancellation by @dranikpg in feat(server): Improved cancellation #599
• feat(replica): atomicity for multi shard commands by @adiholden in feat(replica): atomicity for multi shard commands #598
• fix(server):set of existing object with expiry by @ashotland in fix(server):set of existing object with expiry #609
• chore: Add tiered_storage_test. by @romange in chore: Add tiered_storage_test. #613
• feat(tools):Cache logs player by @ashotland in feat(tools):Cache logs player #614
• fix(tools): cache logs player handle append as redis append by @ashotland in fix(tools): cache logs player handle append as redis append #615
• docs(quickstart): Clarify docker command for macOS by @pjambet in docs(quickstart): Clarify docker command for macOS #612
• feat(tools):cache log player batching all the way optimisation by @ashotland in feat(tools):cache log player batching all the way optimisation #617
• feat(ci test): add testing for helm chart by @tamcore in feat(ci test): add testing for helm chart #622
• chore: overhaul chart by @tamcore in chore: overhaul chart #620
• Cache logs player - add one last stats print after completion by @ashotland in Cache logs player - add one last stats print after completion #623
• feat(server): Buffered journal serializers by @dranikpg in feat(server): Buffered journal serializers #619
• feat(server): Update helio, optimize and clean up rdb/snapshot by @dranikpg in feat(server): Update helio, optimize and clean up rdb/snapshot #625
• fix(server): only print version if successfully connect. by @boazsade in fix(server): only print version if successfully connect. #628
• feat(server): using memory defrag with per shard mem info by @boazsade in feat(server): using memory defrag with per shard mem info #616

New Contributors

• @arukiidou made their first contribution in fix(helm): Chart.yaml - Add home, keywords, sources, kubeVersion #542
• @ashotland made their first contribution in feat(server): track hits and misses stats. #608
• @pjambet made their first contribution in docs(quickstart): Clarify docker command for macOS #612

Full Changelog: v0.12.0...v0.13.0

---

This discussion was created from the release v0.13.0.

[snird]

SSD data-tiering is a killer feature!
Awesome release @romange and the team.