Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include docker-compose steps in hosting for localized database backups #7929

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

aaronkanzer
Copy link

@aaronkanzer aaronkanzer commented Jul 17, 2024

Provides docker-compose steps that allow an end user to easily access and run commands for data integrity and reliability against a local fossil-db and postgres instances

Cc @kabilar

URL of deployed dev instance (used for testing):

N/A

Steps to test:

  • Create a deployment of WebKNOSSOS via instructions on https://docs.webknossos.org/webknossos/installation.html
  • Once all containers are healthly on your local EC2, run docker-compose run --rm fossil-db-backup
  • Confirm backup is present in /persistent/fossildb/backup/private on EC2 instance
  • Run docker-compose run --rm fossil-db-restore to observe successful restoration
    (additionally, you can destroy the local fossildb instance and see if you can re-populate via restore)

TODOs:

  • Include similar docker-compose scripts for pg_dump equivalent

Issues:

  • fixes #

(Please delete unneeded items, merge only when none are left open)

@CLAassistant
Copy link

CLAassistant commented Jul 17, 2024

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ aaronkanzer
❌ Aaron Kanzer


Aaron Kanzer seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@aaronkanzer aaronkanzer marked this pull request as ready for review July 19, 2024 17:49
Copy link
Member

@normanrz normanrz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your PR. I agree that it would be useful to include the backup commands in the docker-compose.yaml. I wonder if we should add a backup.sh to also zip up the fossildb backup with a timestamped filename?

I don't think the explicit networks are required and would prefer to keep them out.

In order to merge PRs from contributors, we need the CLA check to pass. For that all commits need to be authored with matching email info. Sorry for the inconvenience.

@@ -59,6 +61,7 @@ services:
# FossilDB
fossildb:
image: scalableminds/fossildb:master__489
container_name: webknossos-fossildb-1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
container_name: webknossos-fossildb-1

# FossilDB Backup -- backup located to ../persistent/fossildb/backup/private after completion
# Example command: docker-compose run --rm fossil-db-backup
fossil-db-backup:
image: scalableminds/fossildb-client:master
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
image: scalableminds/fossildb-client:master
image: scalableminds/fossildb-client:master__489

depends_on:
fossildb:
condition: service_healthy
command: ["webknossos-fossildb-1", "backup"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
command: ["webknossos-fossildb-1", "backup"]
command: ["fossildb", "backup"]

Comment on lines +32 to +33
networks:
- webknossos_default
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think an explicit network is required. We don't have that in our docker compose-based deployments.

Suggested change
networks:
- webknossos_default

Comment on lines +124 to +125
networks:
- webknossos_default
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
networks:
- webknossos_default

Comment on lines +128 to +129
# Example command: docker-compose run --rm fossil-db-restore
fossil-db-restore:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Example command: docker-compose run --rm fossil-db-restore
fossil-db-restore:
# Example command: docker-compose run --rm fossildb-restore
fossildb-restore:

Comment on lines +117 to +118
# Example command: docker-compose run --rm fossil-db-backup
fossil-db-backup:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Example command: docker-compose run --rm fossil-db-backup
fossil-db-backup:
# Example command: docker-compose run --rm fossildb-backup
fossildb-backup:

volumes:
- ./backups:/backups
depends_on:
- postgres
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- postgres
postgres:
condition: service_healthy


# Postgres backup
postgres-backup:
image: postgres
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
image: postgres
image: postgres:15-bullseye

# Postgres backup
postgres-backup:
image: postgres
command: [ "/bin/bash", "-c", "PGPASSWORD=postgres pg_dump -Fc -h postgres -U postgres webknossos | gzip > /backups/backup_$(date +%Y-%m-%d_%H-%M).sql.gz" ]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
command: [ "/bin/bash", "-c", "PGPASSWORD=postgres pg_dump -Fc -h postgres -U postgres webknossos | gzip > /backups/backup_$(date +%Y-%m-%d_%H-%M).sql.gz" ]
command: [ "/bin/bash", "-c", "pg_dump -Fc -h postgres -U postgres webknossos | gzip > /backups/backup_$(date +%Y-%m-%d_%H-%M).sql.gz" ]
environment:
PGPASSWORD: postgres

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants