Current status

[mrx8]

Is your project ongoing?
There are no updates since last year.
I would be truly sad if it will not be ongoing...
Additionally the current docker image segfaults for me:

docker run \
  --ulimit memlock=-1:-1 \
  --ulimit nofile=262144:262144 \
  -p 6379:6379 \
  -p 6380:6380 \
  -p 9090:9090 \
  -it \
  --rm \
  cachegrand/cachegrand-server:latest
> Generating SSL certificate
--> Certificate Authority
====> Generating new CA key /etc/cachegrand/ca.key
====> Generating new CA Certificate /etc/cachegrand/ca.pem
====> Generating new config file openssl.cnf
--> Certificate
====> Generating new SSL KEY /etc/cachegrand/certificate.key
====> Generating new SSL CSR /etc/cachegrand/certificate.csr
====> Generating new SSL CERT /etc/cachegrand/certificate.pem
[2024-05-07T08:00:00Z][INFO       ][program_startup_report] cachegrand-server version v0.4.0 (built on 2023-05-28T18:17:00Z)
[2024-05-07T08:00:00Z][INFO       ][program_startup_report] > Release build, compiled using gcc v11.3.0
[2024-05-07T08:00:00Z][INFO       ][program_startup_report] > Hashing algorithm in use t1ha2
[2024-05-07T08:00:00Z][INFO       ][program_startup_report] > Running on Linux 08a7c3b776d4 6.1.92-1-MANJARO #1 SMP PREEMPT_DYNAMIC Mon May 27 03:37:00 UTC 2024 x86_64
[2024-05-07T08:00:00Z][INFO       ][program_startup_report] > Memory: 31447 MB total, 0 MB swap total
[2024-05-07T08:00:00Z][INFO       ][program_startup_report] > TLS: mbed TLS 2.28.0 (kernel offloading disabled)
[2024-05-07T08:00:00Z][INFO       ][program_startup_report]        Try to load the tls kernel module with "modprobe tls", no need to restart cachegrand-server
[2024-05-07T08:00:00Z][INFO       ][program_startup_report] > Realtime clock source <POSIX>, resolution <3 ms>
[2024-05-07T08:00:00Z][INFO       ][program_startup_report] > Monotonic clock source <Hardware (TSC)> (estimated cpu cycles per second <3.90 GHz>), resolution <1 ms>
[2024-05-07T08:00:00Z][INFO       ][config] Loading the configuration from /etc/cachegrand/cachegrand.yaml
[2024-05-07T08:00:00Z][WARNING    ][program_ulimit] Unable to set max opened file ulimit
[2024-05-07T08:00:00Z][INFO       ][program] Starting <12> workers
[2024-05-07T08:00:00Z][ERROR      ][worker][id: 00][cpu: 00][io_uring_support] Unable to allocate io_uring using the given params
[2024-05-07T08:00:00Z][ERROR      ][worker][id: 00][cpu: 00][io_uring_support] Unable to allocate io_uring using the given params
[2024-05-07T08:00:00Z][WARNING    ][worker][id: 00][cpu: 00][worker_iouring] io_uring linking not supported, accepting new connections will incur in a performance penalty
[2024-05-07T08:00:00Z][ERROR      ][worker][id: 00][cpu: 00][io_uring_support] Unable to allocate io_uring using the given params
[2024-05-07T08:00:00Z][ERROR      ][worker][id: 00][cpu: 00][worker] io_uring worker initialization failed, terminating
/startup.sh: line 7:    14 Segmentation fault      (core dumped) /usr/bin/cachegrand-server -c /etc/cachegrand/cachegrand.yaml

[danielealbano]

Sorry to read about your issue @mrx8, I haven't had room to work on it in the last few months but it's definitely not dead and the 0.5 release is almost ready including a number of fixes and internal improvements!
I will probably moving very slowly till October :)

I will try to investigate your issue over the weekend but from the logs it seems there might be an issue with the permission / capabilities of accessing io_uring

[danielealbano]

One of the major changes I am working on is to make the internal hashtable operations retry-able, this is specially useful because when the hashtable founds a lock it starts to wait and although normally it doesn't wait too much (unless there are a lot of writes on the same key), it prevents all other connections to be processed wasting time! The new version solves this problem simply having the hashtable informing the upper layer that it can't proceed and it has to retry so the other connections within the same thread are processed instead of just being stuck and waiting.

I am also going to drop the on-disk storage has I have never completed it and needs to be heavily reviewed, so makes no sense to keep it around as it has a direct impact on the performance.

At last, I am going to change the layout of the data in memory as right now the data are chunked in block of 64kb making all the business logic extremely challenging to work with, it was done in this way to support the on-disk storage but as I am dropping that layer for now it also makes no sense to keep this extra complexity in place.
This should actually improve your use case as the system will not need to deal with chunks of data and instead have a contiguous block, dropping a lot of intermediate data structures.

There are honestly a few more bits I want to experiment with but it's mostly changes to the hashtable, how the data are locked or accessed.

I will have more details on the latencies only when I wrap up the changes, I have relocated and had to sell all my benchmarking lab so I am back to do these tests on online providers.

In terms of your use case, do you do mostly just basic GET and SET? If not, can you share a few example of commands? (just the commands with the options, no need to share the key / values).
Do you do often a SET on a key being read? Or is it more a rate occurance?

[simaocat]

Our usage scenario involves a large amount of memory read and write operations, and we will only use the most basic commands, such as: set, get, del, and possibly with ttl (The Redis protocol is actually not needed, on the contrary, it would introduce overhead in parsing, custom protocol may be a better choice). Therefore, having drop on disk storage would be good news, as it would both simplify the code and reduce unnecessary overhead. Our access pattern is to set a key (1-50MB) and then there may be a large number of reads, but we will not repeatedly set the same key (the key is a hash value of uint64 type).

Additionally, I have three more questions:

1. It seems that max_memory_usage only accounts for the size of all values added together and does not consider the memory occupied by keys or structures. This could lead to confusion, and we hope it can be changed to limit the actual maximum memory usage of the process (at least keys and values together), so the lru or lfu eviction could start earlier.
2. Another idea is whether disk storage can serve as a supplement to memory, similar to memcached's extstore. When evictions occur, if disk storage is configured, the evicted values could be placed on the disk.
3. Is there a plan to support RDMA in the future?

[danielealbano]

To be honest I am very interested in supporting RDMA because of the scenarios it opens with NVIDIA GPUDirect RDMA, I have some of the hardware needed at home (a few Mellanox ConnectX-5 QSFP28 and a switch that supports RDMA that runs sonic, but I lent one of my EPYC servers and sold the rest so can't do local development or testing, also I need to silence the switch ;)) but I don't have a GPU that supports GPUDirect (I was thinking to get an NVIDIA V100 that should support it but I haven't explored it in detail yet).

I am painfully aware that TCP/IP + the Redis protocol is the killer (in the bad sense) here, I was thinking to add support for an UDP-based message-oriented protocol (with of course the necessary bits to ensure re-sends if data are lost) because there is too much overhead with TCP/IP and also it's sequential, can't send just a flurry of parts in parallel without adding more overhead. Probably supporting RDMA + an UDP message oriented protocol (with the necessary client libraries) would be the right choice for most enterprise use cases.

RDMA is really cool but even google is switching away, slowly, because of the extra complexity it brings in, however NVIDIA GPUDirect RDMA is very cool and I really want to support it.

But these are ideas for the future, not something I am planning to implement right now as cachegrand is my side project not my main activity :)

It's great at hear that the proportion GET / SET is greatly oriented to the GETs, probably some of the changes I want to test out might also help in that direction: right now when a SET is done, the system will still be able to provide the previous data until the new data is fully committed but this introduces so many layers of complexity, I am planning to run a test using a custom rwlock implementation for fibers and see how the performances are impacted (among other things this will reduce the amount of actual locking needed when multiple writes for different keys end up in the same sub-segment of the hashtable).

About the other two questions:

1. I will review max_memory_usage to include the space of the keys, I don't want to start to bring in the data structurs as these might change at the moment making the accounting code much more complex to maintain; however bringing in the keys will fill a major gap
2. Initially I wanted to implement a tiered storage so that data can be read from disk if they are not in memory, I am not throwing away the general idea just the current implementation as it's way over complex for what's actually needed.

[simaocat]

Sounds great! Thank you very much! I am very much looking forward to your new version, the sooner, the better!

[danielealbano]

I still have to wrap up some important bits and then do some validation via valgrind understress. Will take a while but once I have some of the changes merged I will send you a build!