Skip to content

Commit 19a5aaa

Browse files
committed
Update README
1 parent 18c6abd commit 19a5aaa

File tree

2 files changed

+223
-20
lines changed

2 files changed

+223
-20
lines changed

README.md

+223-20
Original file line numberDiff line numberDiff line change
@@ -1,35 +1,238 @@
1-
# KiWi
1+
<p align="center">
2+
<img src="./docs/logo.png" alt="KiWi Logo" width="200" height="220"/>
3+
</p>
24

3-
KiWi is a simple key-value store that supports the following operations:
5+
# KiWi: A Lightweight RESP Key-Value Store.
46

5-
- `PUT <key> <value>`: Store the given value under the given key.
6-
- `GET <key>`: Retrieve the value stored under the given key.
7-
- `DELETE <key>`: Remove the value stored under the given key.
7+
KiWi is a high-performance, [RESP](https://redis.io/docs/latest/develop/reference/protocol-spec/)
8+
compliant, key-value store inspired by the Bitcask paper, designed
9+
for simplicity, reliability, and blazing-fast read/write operations.
810

9-
## Build
11+
## Features
1012

11-
To build the project, run the following command:
13+
- RESP protocol support for Redis-compatible client interaction.
14+
- Non-blocking I/O server with Netty.
15+
- High-performance key-value store based on the Bitcask storage model.
16+
- In-memory indexing for fast reads.
17+
- TTL-based key expiration.
18+
- Checksums for data integrity.
19+
- Compaction and efficient file merging process.
20+
- Hint files for quick startup times.
21+
- Tunable durability.
1222

13-
```shell
14-
./gradlew build
15-
```
23+
## Quick Start
24+
25+
1. Start Docker container
26+
```bash
27+
docker run --rm --name kiwi -p 6379:6379 nemanjam/kiwi:latest
28+
```
29+
30+
2. Connect to the server with `redis-cli`
31+
```bash
32+
redis-cli -h localhost
33+
```
34+
35+
3. Use the server as you would a Redis server.
36+
```text
37+
SET key value
38+
OK
39+
40+
GET key
41+
"value"
42+
43+
EXISTS key
44+
(integer) 1
45+
46+
DEL key
47+
OK
48+
49+
EXISTS key
50+
(integer) 0
51+
```
52+
53+
### Supported Commands
54+
55+
- `SET key value`
56+
- `GET key`
57+
- `DEL key`
58+
- `EXISTS key`
59+
- `FLUSHDB`
60+
- `PING`
61+
- `DBSIZE`
62+
- `INFO`
63+
64+
## Installation
65+
66+
### Prerequisites
67+
68+
- Java 21
69+
- Docker (optional, for running via a container)
70+
71+
### From Source
72+
73+
1. Clone the repository:
74+
```bash
75+
git clone https://github.com/nemanjam/kiwi.git
76+
cd kiwi
77+
```
78+
2. Build the project:
79+
```bash
80+
./gradlew assembleDist installDist
81+
```
82+
3. Run the KiWi server:
83+
```bash
84+
./kiwi-server/build/install/kiwi-server/bin/kiwi-server
85+
```
86+
87+
### Docker
88+
89+
1. Build the Docker image
90+
```bash
91+
docker build -t kiwi .
92+
```
93+
2. Run the container:
94+
```bash
95+
docker run --rm --name kiwi -p 6379:6379 kiwi:latest
96+
```
97+
3. Connect to the server:
98+
```bash
99+
redis-cli -h localhost -p 6379
100+
```
101+
102+
## Configuration
103+
104+
KiWi can be configured using environment variables or a HOCON configuration file.
105+
Refer to [Typesafe Config](https://github.com/lightbend/config) for configuration examples.
106+
107+
Default values are:
108+
109+
- Storage [application.conf](kiwi-core/src/main/resources/application.conf)
110+
- Server [application.conf](kiwi-server/src/main/resources/application.conf)
111+
112+
## Benchmarks
113+
114+
KiWi can be evaluated with [redis-benchmark](https://redis.io/topics/benchmarks) utility command.
115+
116+
Below are the results of running `redis-benchmark` with KiWi and Redis on a local setup (MacBook M3
117+
Pro with 18GB RAM and Sequoia 15.1.1).
16118

17-
## Test
119+
```text
120+
redis-benchmark -h localhost -t set -n 100000 -r 10000000 -d 1024
18121
19-
To execute the tests, use the following command:
122+
====== SET ======
123+
100000 requests completed in 2.18 seconds
124+
50 parallel clients
125+
1024 bytes payload
126+
keep alive: 1
127+
host configuration "save":
128+
host configuration "appendonly":
129+
multi-thread: no
20130
21-
```shell
22-
./gradlew test
131+
Summary:
132+
throughput summary: 45934.77 requests per second
133+
latency summary (msec):
134+
avg min p50 p95 p99 max
135+
1.016 0.088 0.951 1.863 2.703 30.655
23136
```
24137

25-
## Run
138+
```text
139+
redis-benchmark -h localhost -t get -n 100000 -r 10000000 -d 1024
26140
27-
To run the project, execute the following commands:
141+
====== GET ======
142+
100000 requests completed in 1.83 seconds
143+
50 parallel clients
144+
1024 bytes payload
145+
keep alive: 1
146+
host configuration "save":
147+
host configuration "appendonly":
148+
multi-thread: no
28149
29-
```shell
30-
./gradlew assembleDist installDist
150+
Summary:
151+
throughput summary: 60753.34 requests per second
152+
latency summary (msec):
153+
avg min p50 p95 p99 max
154+
0.661 0.088 0.639 0.967 1.311 12.255
31155
```
32156

33-
```shell
34-
./build/install/kiwi/bin/kiwi
157+
JVM options:
158+
159+
```text
160+
-Xms2g -Xmx2g -XX:UseG1GC –XX:+UseStringDeduplication -XX:+AlwaysPreTouch
161+
````
162+
163+
KiWi Configuration:
164+
165+
```hocon
166+
kiwi {
167+
storage {
168+
log {
169+
dir = "/tmp/kiwi"
170+
segment.bytes = 1073741824 // 1GB
171+
172+
sync {
173+
mode = "periodic"
174+
periodic {
175+
interval = 10s
176+
}
177+
}
178+
}
179+
}
180+
}
35181
```
182+
183+
## Design
184+
185+
KiWi combines the simplicity of RESP with the efficient storage model described in the Bitcask
186+
paper. This architecture is designed for high performance and simplicity.
187+
188+
### Storage Model
189+
190+
- All write operations are appended to a log file, ensuring sequential disk writes for maximum
191+
performance.
192+
- When the active log file reaches a configurable size, it is rolled over to a segment file.
193+
- Periodically, segment files are compacted to remove stale data and reclaim disk space.
194+
- Crash recovery is achieved by replaying the log files during startup.
195+
- Disk I/O operations, like log compaction, are handled in background threads to avoid blocking
196+
client requests.
197+
198+
### In-Memory Index
199+
200+
- All keys are stored in an in-memory hash table, pointing to their location in the log file.
201+
- This ensures `O(1)` read performance while keeping the storage footprint minimal.
202+
203+
### Non-Blocking I/O Server
204+
205+
- Netty-based event loop for handling client requests.
206+
- KiWi supports the RESP protocol, making it compatible with Redis clients and tools.
207+
208+
### Durability
209+
210+
- KiWi provides tunable durability options to balance performance and data safety:
211+
- `periodic` (default): Writes are flushed to disk at regular intervals.
212+
- `batch`: Writes are batched and flushed when the batch window expires. All writers are blocked
213+
until the batch is written.
214+
- `lazy`: Flush is delegated to the operating system, which may delay writes for performance.
215+
216+
### Pros
217+
218+
- Fast writes due to sequential disk I/O.
219+
- Fast reads with O(1) lookups using the in-memory index.
220+
- Simple and robust crash recovery with the data and hint files.
221+
- Incremental crash-safe compaction process.
222+
223+
### Cons
224+
225+
- The in-memory index requires all keys to fit in memory.
226+
- Log compaction introduces periodic I/O overhead.
227+
228+
## Contributing
229+
230+
We welcome contributions to KiWi! Here’s how you can help:
231+
232+
1. Fork the repository.
233+
2. Create a new branch for your feature or bugfix.
234+
3. Submit a pull request with a clear description of your changes.
235+
236+
## License
237+
238+
KiWi is licensed under the MIT License. See [LICENSE](./LICENSE) for details.

docs/logo.png

225 KB
Loading

0 commit comments

Comments
 (0)