How to configure row cache value to improve read performance?

[GoodManWEN]

I read in the configuration file that if the use case is mostly point-like kv search, then enabling row cache may help improve performance. This happens to be my use case exactly, but unfortunately, there is no example given in the configuration file, so I don’t know how to enable this option appropriately.

I have 64GB of physical memory and have already allocated 24GB of rocksdb.subkey_block_cache_size and 8GB of rocksdb.metadata_block_cache_size. I’m not sure what scenarios they are responsible for and how to allocate the proportions. Any suggestions from our fellow community?

[git-hulk]

Hi @GoodManWEN

We have enabled the configuration https://github.com/apache/incubator-kvrocks/blob/unstable/kvrocks.conf#L530 by default which means those two configurations will share the memory cache. And we still keep them instead of the single one for compatibility reasons.

[GoodManWEN]

Thank you @git-hulk . So actually it doesn’t matter what their allocation ratio is, as long as the total amount is the same, is that what it means?

In the test I conducted a few days ago, I tested about 100 million keys of random data, and each value was about 16KB in length. I configured a total of 32GB maximum cache of 64GB for the above two items. During global random reading test, the memory usage has been maintained at around 8G, which I think is a relatively low level and makes me doubt whether the memory has been fully utilized (of course, the system cache has almost filled up all 64GB). According to the optimization suggestions I saw in on of the rocksdb tuning document before, it suggests setting the block cache to one-third of the physical memory and leaving the rest to the system cache. I am not sure if his wording refers to setting the upper limit in the setting item to one-third or whether one-third of physical memory should be occupied during actual operation. It would be very helpful if you could provide some additional explanations.

[git-hulk]

Can you check if rocksdb.share_metadata_and_subkey_block_cache was enabled?

[GoodManWEN]

Yes, this option is enabled. So from your reply, I deduce that after enabling this option, the sum of the two(metadata & subkey cache size) should be close to the memory occupied by the application under normal circumstances, my situation is a bit incorrect?

[GoodManWEN]

Btw, could you please provide a configuration example of the option in this link https://github.com/apache/incubator-kvrocks/blob/unstable/kvrocks.conf#L537. Due to the lack of examples, I am not sure what unit this option is in. Should I set it to 1, or several bytes, or several GB? In addition, about row cache, it seems that this is not a native rocksDB function, but a feature of kvrocks. I don’t know if there is a recommended size for setting it. What is the appropriate physical memory usage ratio for multi-read and few-write scenarios?

[git-hulk]

Yes, this option is enabled. So from your reply, I deduce that after enabling this option, the sum of the two(metadata & subkey cache size) should be close to the memory occupied by the application under normal circumstances, my situation is a bit incorrect?

That's right. The block size should roughly reach the sum of those two configurations if the dataset is large enough.

[git-hulk]

Due to the lack of examples, I am not sure what unit this option is in. Should I set it to 1, or several bytes, or several GB?

Good catch, we should describe the unit in the configuration file, the unit is MiB indeed.