About Lua atomicity in Kvrocks

[yp963]

I’ve observed an interesting behavior regarding Lua script execution in Kvrocks. When a Lua script contains an infinite loop, operations before the loop (e.g., SET) are persisted, while subsequent operations are not executed. After a restart, I can still retrieve the values set before the loop (e.g., GET 'a' returns 10 in the example below).

redis.call('SET', 'a', '10')

local a_value = redis.call('GET', 'a')

if a_value == '10' then
    while true do
        redis.log(redis.LOG_WARNING, "Looping indefinitely...")
    end
    redis.call('SET', 'a', '20')
else
    redis.call('SET', 'a', '30')
end

return "Script finished (should not happen if 'a' was 10)"

This differs from Redis.

I’m curious about the design choices behind this behavior. Why wasn't RocksDB's transaction feature used to enforce full atomicity? Was this a performance optimization, or were there other trade-offs?

[git-hulk]

I’m curious about the design choices behind this behavior. Why wasn't RocksDB's transaction feature used to enforce full atomicity? Was this a performance optimization, or were there other trade-offs?

@yp963 It's not a performance consideration, because read-write Lua script is a single-thread mode in Kvrocks. We could enhance this by enabling the transaction mode for Lua like MULTI-EXEC command as well.

[yp963]

Thank you, but what I mean is: how does kvrocks ensure data consistency after restart when the system crashes or the Lua script is in an infinite loop?

[PragmaTwice]

Sorry I cannot get your point.

In computer programs, an infinite loop indicates that the following code is all unreachable.

For example, a program

while true { ... } // no break in this block
do_something()

is equal to

while true  { ... }
do_nothing() // unreachable!

regardless of whether do_something has any side effect or not.

[yp963]

Sorry about that.
What I mean is: assuming it's not an infinite loop, but a crash.
For example,
set a = 10;
crash
{do_anything};
In this case, after a restart,Redis set a=10 would not take effect, but in KvRocks, it would.
In this scenario, doesn't kvrocks guarantee the atomicity of the lua script?

[yp963]

As I understand it, redis writes the entire lua script into wal (aof mode) after it is executed, and kvrocks writes each set operation in the lua script into the rocksdb wal, which leads to a restart due to downtime or other reasons before the lua script is completed.
In this case, the results of kvrocks are different from those of redis, right?

[PragmaTwice]

We have an experimental config option txn-context-enabled for this.