Replies: 1 comment
-
I cannot think of a better idea to improve this situation at the moment. But I feel it's too tricky to allow disabling TTL for a single type. And a possible solution from the user side is to use HASH instead of STRING, so that it could filter the blob value with its key. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
I was benchmarking Kvrocks' write performance with / without blob files my timing how long it takes for 10 clients to write 20GB of 4mb strings. Even though all of the strings are large enough to be stored in blob files which should improve performance, enabling blob files actually reduced the performance by around 3x (194 seconds vs 600 seconds for 20GB of write).
I tried profiling the code with perf and found this:

Turns out expiration timestamps are stored in the value which in this case are within the blob files. This means that for engine::MetadataFilter, we need to do a disk read for every ke which is a lot of overhead. I tried disabling TTL with the snippets below and was able to improve the write performance to 78 seconds:
I think this is a nice improvement to have since it increases write throughput with blob files by almost 10x. I thought of a couple potential implementation methods and wanted to get your thoughts.
My initial idea is to store expiration timestamp with the key. This is pretty unrealistic since every database created before the change is no longer compatible. A second approach could be to add a configuration option to disable TTL. This would involve disabling the metadata filter and rejecting Redis 'SET' requests that include expiration commands.
TTL is pretty fundamental to Redis, which makes this pretty challenging to integrate. I want to hear about your opinions on these ideas or any other potential paths forward, and whether you feel this is something we should incorporate.
Beta Was this translation helpful? Give feedback.
All reactions