Feature: Add Snapshot functionality to Rust SDK and CLI#119
Draft
psrvere wants to merge 1 commit intotursodatabase:mainfrom
Draft
Feature: Add Snapshot functionality to Rust SDK and CLI#119psrvere wants to merge 1 commit intotursodatabase:mainfrom
psrvere wants to merge 1 commit intotursodatabase:mainfrom
Conversation
Implements consistent point-in-time snapshot API in the rust SDK - discovers and creates tables and indexes from sqlite_schema - clones data for each table - fsync is switched off for faster writes Signed-off-by: Prateek Singh Rathore <prateek.singh.rathore@gmail.com>
Collaborator
|
Hey @psrvere, interesting approach. The idea here is that the immediate transaction ensures that there are no new WAL entries, which seems to be safe. However, I suspect this is going to be pretty slow for larger databases because it's all going through the SQL interface as far as I can tell. IMHO, a better long-term strategy is to do this in turso.git and just make a copy of the database file itself while blocking concurrent writers. |
Contributor
Author
|
Thanks for the feedback @penberg. Yes, copying the file would be much faster. I saw a similar SQL based implementation in turso CLI and hence assumed that if its good there, it should be good in AgentFS. Let me do a file copy snapshot implementation in turso and share a draft PR soon. |
psrvere
added a commit
to psrvere/turso
that referenced
this pull request
Dec 24, 2025
This addresses feedback from tursodatabase/agentfs#119 suggesting that snapshot functionality should be implemented in turso.git using direct fily copying rather than SQL-based row by row copying. Comment: https:://github.com/tursodatabase/agentfs/pull/119#issuecomment-3681336678 - extract core logic from Pager::checkpoint function to a new Pager::checkpoing_internal function and add a flag to keep_lock during the Finalize phase - create wrapper functions Pager::checkpoint, Pager::checkpoint_with_lock and Pager::block_checkpoint_keep_lock - add Connection::snapshot API that checkpoints while keeping the lock and copies the database file The lock is held to avoid a race condition after finishing checkpointing and before copying the file when concurrent writers can write to db. Signed-off-by: Prateek Singh Rathore <prateek.singh.rathore@gmail.com>
psrvere
added a commit
to psrvere/turso
that referenced
this pull request
Dec 24, 2025
This addresses feedback from tursodatabase/agentfs#119 suggesting that snapshot functionality should be implemented in turso.git using direct fily copying rather than SQL-based row by row copying. Comment: https:://github.com/tursodatabase/agentfs/pull/119#issuecomment-3681336678 - extract core logic from Pager::checkpoint function to a new Pager::checkpoing_internal function and add a flag to keep_lock during the Finalize phase - create wrapper functions Pager::checkpoint, Pager::checkpoint_with_lock and Pager::block_checkpoint_keep_lock - add Connection::snapshot API that checkpoints while keeping the lock and copies the database file The lock is held to avoid a race condition after finishing checkpointing and before copying the file when concurrent writers can write to db. Signed-off-by: Prateek Singh Rathore <prateek.singh.rathore@gmail.com>
Contributor
Author
|
@penberg - I created tursodatabase/turso#4346 PR for adding snapshot API to Connection. Please review. |
psrvere
added a commit
to psrvere/turso
that referenced
this pull request
Dec 26, 2025
This addresses feedback from tursodatabase/agentfs#119 suggesting that snapshot functionality should be implemented in turso.git using direct fily copying rather than SQL-based row by row copying. Comment: https:://github.com/tursodatabase/agentfs/pull/119#issuecomment-3681336678 - extract core logic from Pager::checkpoint function to a new Pager::checkpoing_internal function and add a flag to keep_lock during the Finalize phase - create wrapper functions Pager::checkpoint, Pager::checkpoint_with_lock and Pager::block_checkpoint_keep_lock - add Connection::snapshot API that checkpoints while keeping the lock and copies the database file The lock is held to avoid a race condition after finishing checkpointing and before copying the file when concurrent writers can write to db. Signed-off-by: Prateek Singh Rathore <prateek.singh.rathore@gmail.com>
psrvere
added a commit
to psrvere/turso
that referenced
this pull request
Jan 5, 2026
This addresses feedback from tursodatabase/agentfs#119 suggesting that snapshot functionality should be implemented in turso.git using direct fily copying rather than SQL-based row by row copying. Comment: https:://github.com/tursodatabase/agentfs/pull/119#issuecomment-3681336678 - extract core logic from Pager::checkpoint function to a new Pager::checkpoing_internal function and add a flag to keep_lock during the Finalize phase - create wrapper functions Pager::checkpoint, Pager::checkpoint_with_lock and Pager::block_checkpoint_keep_lock - add Connection::snapshot API that checkpoints while keeping the lock and copies the database file The lock is held to avoid a race condition after finishing checkpointing and before copying the file when concurrent writers can write to db. Signed-off-by: Prateek Singh Rathore <prateek.singh.rathore@gmail.com>
Contributor
Author
|
@penberg - I am not able to get reviews/feedback on snapshot PR - tursodatabase/turso#4346 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #57
Closes #61
Implements consistent point-in-time snapshot API in the rust SDK