Open
Description
I am using litestream to replicate the local database of my Python application running in a container to an SFTP server.
/etc/litestream.yml
dbs:
- path: database.sqlite3
replicas:
- type: sftp
host: sftp.fr.company.com
user: XXX
password: XXX
path: /backup/app/${LITESTREAM_REPLICA_PATH}
retention: 168h # 7 days
snapshot-interval: 24h
# The 33m is to prevent snapshot and validations from happening concurrently.
# https://github.com/benbjohnson/litestream/issues/253
validation-interval: 12h33m
Here is my application service startup log:
+ litestream restore -if-replica-exists database.sqlite3
time=2025-01-13T18:57:39.516+01:00 level=INFO msg="restoring snapshot" db=/home/app/database.sqlite3 replica=sftp generation=9d7f0b28cf1c6440 index=0 path=/home/app/database.sqlite3.tmp
time=2025-01-13T18:57:39.524+01:00 level=INFO msg="restoring wal files" db=/home/app/database.sqlite3 replica=sftp generation=9d7f0b28cf1c6440 index_min=0 index_max=0
time=2025-01-13T18:57:39.528+01:00 level=INFO msg="downloaded wal" db=/home/app/database.sqlite3 replica=sftp generation=9d7f0b28cf1c6440 index=0 elapsed=4.470316ms
time=2025-01-13T18:57:39.530+01:00 level=INFO msg="applied wal" db=/home/app/database.sqlite3 replica=sftp generation=9d7f0b28cf1c6440 index=0 elapsed=2.073276ms
time=2025-01-13T18:57:39.530+01:00 level=INFO msg="renaming database from temporary location" db=/home/app/database.sqlite3 replica=sftp
+ alembic upgrade head
INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
INFO [sqlalchemy.engine.Engine] BEGIN (implicit)
2025-01-13 18:57:41,348 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2025-01-13 18:57:41,348 INFO sqlalchemy.engine.Engine PRAGMA main.table_info("alembic_version")
2025-01-13 18:57:41,348 INFO sqlalchemy.engine.Engine [raw sql] ()
INFO [sqlalchemy.engine.Engine] PRAGMA main.table_info("alembic_version")
INFO [sqlalchemy.engine.Engine] [raw sql] ()
2025-01-13 18:57:41,349 INFO sqlalchemy.engine.Engine SELECT alembic_version.version_num
FROM alembic_version
INFO [sqlalchemy.engine.Engine] SELECT alembic_version.version_num
FROM alembic_version
2025-01-13 18:57:41,349 INFO sqlalchemy.engine.Engine [generated in 0.00015s] ()
INFO [sqlalchemy.engine.Engine] [generated in 0.00015s] ()
2025-01-13 18:57:41,354 INFO sqlalchemy.engine.Engine ROLLBACK
INFO [sqlalchemy.engine.Engine] ROLLBACK
+ exec litestream replicate -exec ./run.py
time=2025-01-13T18:57:41.481+01:00 level=INFO msg=litestream version=v0.3.13
time=2025-01-13T18:57:41.482+01:00 level=INFO msg="initialized db" path=/home/app/database.sqlite3
time=2025-01-13T18:57:41.482+01:00 level=INFO msg="replicating to" name=sftp type=sftp sync-interval=1s host=sftp.fr.company.com user=XXX path=/backup/app/staging/demo
time=2025-01-13T18:57:42.485+01:00 level=INFO msg="sync: new generation" db=/home/app/database.sqlite3 generation=e82baa425de42b74 reason="no generation exists"
time=2025-01-13T18:57:42.501+01:00 level=INFO msg="write snapshot" db=/home/app/database.sqlite3 replica=sftp position=e82baa425de42b74/00000000:4152
time=2025-01-13T18:57:42.509+01:00 level=INFO msg="snapshot written" db=/home/app/database.sqlite3 replica=sftp position=e82baa425de42b74/00000000:4152 elapsed=7.576832ms sz=2840
time=2025-01-13T18:57:42.511+01:00 level=INFO msg="write wal segment" db=/home/app/database.sqlite3 replica=sftp position=e82baa425de42b74/00000000:0
time=2025-01-13T18:57:42.517+01:00 level=INFO msg="wal segment written" db=/home/app/database.sqlite3 replica=sftp position=e82baa425de42b74/00000000:0 elapsed=5.298369ms sz=4152
Litestream successfully restores the database with generation 9d7f0b28cf1c6440
, but then it creates a new generation e82baa425de42b74
when the replication process starts (reason="no generation exists"
).
Since the migration process between the restore / replicate steps is not doing anything here, I was expecting the same initial generation 9d7f0b28cf1c6440
to be used as the database file is the same.
Is my understanding correct ?
Metadata
Metadata
Assignees
Labels
No labels