transaction commit doesn't persist to disk

[maan2003]

More details about the bug

at a5230af

[maan2003@runner-01:~/src/tonbo]$ cargo r --example persistence_test -- write --id 0 --value foo
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.20s
     Running `target-nix/debug/examples/persistence_test write --id 0 --value foo`
Opening database at: ./test_db
Starting transaction...
Inserting record: id=0, value=foo
Committing transaction...
Transaction committed successfully!
now exiting

[maan2003@runner-01:~/src/tonbo]$ cargo r --example persistence_test -- read --id 0
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.19s
     Running `target-nix/debug/examples/persistence_test read --id 0`
Opening database at: ./test_db
Looking for record with id=0...
Record with id=0 NOT FOUND!

What is your expected behavior?

second command to returning the value foo

How to reproduce?

see above

[linear]

TON-285 transaction commit doesn't persist to disk

[crwen]

Tonbo use wal buffer by default. If you don't want to lose data, you can change the buffer size to 0:

let options = DbOption::new(Path::from_filesystem_path(&cli.db_path)?, &TestRecordSchema)
    .wal_buffer_size(0);

Or call db.flush_wal to flush wal buffer.

[maan2003]

@crwen I tried calling flush_wal on every commit.
but flush blocks until there are no active transaction which doesn't work for our usecase (we have many long lived transactions).

new repro at 4605726

[maan2003@runner-01:~/src/tonbo]$ cargo r --example persistence_test -- write --id 0 --value foo
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.21s
     Running `/home/maan2003/src/fedimint/target-nix/debug/examples/persistence_test write --id 0 --value foo`
Opening database at: ./test_db
Spawning long-lived transaction that will hold read lock for 30 seconds...
Long-lived transaction created, sleeping for 30 seconds...
Starting main transaction...
Inserting record: id=0, value=foo
Committing transaction...
Transaction committed successfully!
Now attempting to flush WAL (this should block until the long-lived transaction releases the read lock)...
Long-lived transaction waking up and dropping...
Long-lived transaction dropped
WAL flushed successfully after 29.00 seconds
Waiting for long-lived transaction to complete...
now exiting

see how wal flush is blocked for 29 seconds.

[crwen]

flush_wal need to hold write lock of db, so it has to wait other transactions to finish. But I think shared lock is sufficient because wal file lock need to be held during writing.

you can see the changes