Feature: Add Snapshot for point-in-time database copy #4346
Conversation
psrvere
force-pushed
the
snapshot-file-copy
branch
from
65a4d56 to
27a9bcc
Compare
psrvere
requested review from
jussisaurio,
penberg and
pereman2
as code owners
![turso-bot[bot]](https://avatars.githubusercontent.com/in/2434523?s=60&v=4){kind=small}
psrvere
force-pushed
the
snapshot-file-copy
branch
2 times, most recently
from
faa0e3e to
510cb7b
Compare
penberg
changed the title
sivukhin
left a comment
sivukhin
left a comment
There was a problem hiding this comment.
I have some suggestion and concerns regarding current tsnapshot logic.
Also, it will be great to use new snapshotting logic in the .clone CLI command
core/storage/pager.rs
Outdated
| res.release_guard(); | ||
| // Release checkpoint guard if lock is not to be kept | ||
| if !keep_lock { | ||
| res.release_guard(); |
There was a problem hiding this comment.
I don't think that checkpoint guarantee that guard always will be set.
There are cases (for example, empty WAL - so nothing to checkpoint) - where checkpoint will return without holding any lock.
So, this fix do not prevent race condition situation in all situations, IIUC.
There was a problem hiding this comment.
Yeah you are right. I didn't want to touch the checkpoint function in the first place.
core/lib.rs
Outdated
| let pager = self.pager.load(); | ||
| let result = (|| -> Result<()> { | ||
| // Checkpoint and keep the lock | ||
| let _res = pager.blocking_checkpoint_keep_lock( |
There was a problem hiding this comment.
Can we instead of patching checkpoint do snapshot like this:
- Execute truncate checkpoint
- Start read transaction (either explicitly with
BEGINor maybe by just issuingread_txon WAL) - Do the copy of DB file
- End read transaction
There was a problem hiding this comment.
I like this solution. I will implement this.
Although it does leave a tiny race window between truncating file and taking read lock where a writer can write and snapshot is taken. Maybe we can live with this right now. I will leave a detailed comment just in case we see a bug later on.
There was a problem hiding this comment.
tiny race window between truncating file and taking read lock where a writer can write and snapshot is taken
Ah, yeah. The problem here actually is not in writes specifically - but in a checkpoint that can happen in the middle.
Alternative suggestion from me is to start read transaction before checkpoint but issue FULL checkpoint instead of TRUNCATE. This will make it possible for checkpoint to succeed - but also we will hold WAL and prevent any further checkpoint from happening (even if writes will happen).
Also, we should be careful with deferred nature of transaction and maybe we need to execute something after BEGIN statement in order to properly initiate read transaction.
As we are writing database - its better to be safe than sorry :) So even tiny race window is bad actually.
|
Thanks for the feedback @sivukhin. |
This addresses feedback from tursodatabase/agentfs#119 suggesting that snapshot functionality should be implemented in turso.git using direct fily copying rather than SQL-based row by row copying. Comment: https:://github.com/tursodatabase/agentfs/pull/119#issuecomment-3681336678 - extract core logic from Pager::checkpoint function to a new Pager::checkpoing_internal function and add a flag to keep_lock during the Finalize phase - create wrapper functions Pager::checkpoint, Pager::checkpoint_with_lock and Pager::block_checkpoint_keep_lock - add Connection::snapshot API that checkpoints while keeping the lock and copies the database file The lock is held to avoid a race condition after finishing checkpointing and before copying the file when concurrent writers can write to db. Signed-off-by: Prateek Singh Rathore <[email protected]>
Signed-off-by: Prateek Singh Rathore <[email protected]>
The previous implementation attempted to hold checkpoint locks during file copy by adding a keep_lock param to the checkpoint function. However, checkpoint can have multiple early exit paths like empty WAL where no lock is acquired. Also, it clutters the checkpoint function. This implementation executes TRUNCATE checkpoint and then acquires read lock on WAL before copying database file. This ensures new data is written to WAL and new Checkpoint can not be taken before this read lock is released. Signed-off-by: Prateek Singh Rathore <[email protected]>
psrvere
force-pushed
the
snapshot-file-copy
branch
from
525d2ec to
7e71044
Compare
psrvere
changed the title
Signed-off-by: Prateek Singh Rathore <[email protected]>
2 participants
This PR addresses feedback from tursodatabase/agentfs#119 suggesting that snapshot functionality should be implemented in turso.git using direct file copying rather than SQL-based row by row copying. Comment
Implementation:
Connection::snapshotAPI that checkpoints while keeping the lock and copies the database file.cloneuses this snapshot APIAssisted by: Cursor + Claude