Have you considered introducing B+ tree(For example, wiredtiger) as the storage engine?

[y123456yz]

Have you considered introducing B+ tree as the storage engine? This can meet the performance requirements of kvrocks in different scenarios.
In the actual test, it was found that rocksdb has the problem of read-write amplification, especially when the data volume is very large. I am researching wiredtiger(the default storage engine of MongoDB). wiredtiger is an excellent storage engine. Compared with rocksdb, wiredtiger has better performance in some scenarios and also has full-chain diagnostic capabilities. I have been researching wiredtiger for many years, am familiar with the source code implementation of wiredtiger's core modules, and have contributed multiple PR to wiredtiger.

If kvrocks has this idea, I would be very willing to complete the development of the wiredtiger plugin supported by kvrocks on my spare weekends.

Thank you. Looking forward to your recovery.

[git-hulk]

Hi @y123456yz Thanks for proposing this. Yes, we have ever discussed about if it's possible to support multiple underlying engines. The biggest obstacle is that Kvrocks's codebase is deeply binding with RocksDB, and it may need a lot of effort to achieve this goal.

If kvrocks has this idea, I would be very willing to complete the development of the wiredtiger plugin supported by kvrocks on my spare weekends.

From my perspective, I appreciate your interest, and proposals/PRs are always welcome. cc @apache/kvrocks-committers

[PragmaTwice]

It seems hard to introduce an abstract layer for different underlying storage engines (also to manage between optional dependencies). But look forward to your effort.

[y123456yz]

hi, @git-hulk
Technically, I think I can complete this task. Right now, what I'm most concerned about is the wiredtiger license. I'm not quite sure if the wiredtiger license meets the requirements of apache. The following is the acceptance of the wiredtiger license. Please help me assess whether you can accept wiredtiger. If so, I will spend time on this later:

https://source.wiredtiger.com/license.html

[git-hulk]

The complete WiredTiger software package is Open Source software: you are welcome to modify and redistribute it under the terms of version 2 or version 3 of the GNU General Public License as published by the Free Software Foundation.

According to the ASF 3rd Party License Policy[1], GPL licenses are located in category x[2]. So it's disallowed to be introduced as a dependency. cc @PragmaTwice

[1] https://www.apache.org/legal/resolved.html
[2] https://www.apache.org/legal/resolved.html#category-x

[y123456yz]

The complete WiredTiger software package is Open Source software: you are welcome to modify and redistribute it under the terms of version 2 or version 3 of the GNU General Public License as published by the Free Software Foundation.

According to the ASF 3rd Party License Policy[1], GPL licenses are located in category x[2]. So it's disallowed to be introduced as a dependency. cc @PragmaTwice

[1] https://www.apache.org/legal/resolved.html [2] https://www.apache.org/legal/resolved.html#category-x

@git-hulk
I am sorry that wiredtiger does not meet the license requirements.

In addition, I have a question. Why does rocksdb (dual-licensed under both the GPLv2) meet the requirements

rocksdb licence: （dual-licensed under both the GPLv2）
RocksDB is dual-licensed under both the GPLv2 (found in the COPYING file in the root directory) and Apache 2.0 License (found in the LICENSE.Apache file in the root directory). You may select, at your option, one of the above-listed licenses.

thanks.

[PragmaTwice]

You may select, at your option, one of the above-listed licenses.

Just like you mentioned, we choose Apache-2.0 as the license of rocksdb so that we don't need to follow the GPL rules.

Also, that's the meaning of "dual" licenses.

[y123456yz]

You may select, at your option, one of the above-listed licenses.

Just like you mentioned, we choose Apache-2.0 as the license of rocksdb so that we don't need to follow the GPL rules.

Also, that's the meaning of "dual" licenses.

@PragmaTwice
Thank you for your reply. Best wishes.
thanks.

[mapleFU]

I'm general -1 with this without any real code implemented.

Generally rocksdb might not fit all scenerio, but it can be tunned to fit the most cases. And concepts like Snapshot, Compaction Filter, backup is currently strongly relies on the LSM abstraction. It would be a little tricky to introduce advance features using different abstractions.