generated from CDCgov/template
-
Notifications
You must be signed in to change notification settings - Fork 2
Open
Labels
Description
🚧 What needs to be done
The refiner UI allows users to upload an eCR and then download the refined files that are produced. However, there are several issues with the current implementation that need to be addressed so that this feature can continue to exist and be usable in a production environment.
🤔 Why it needs to be done
In order to make these feature resilient and reliable, we should store the generated files in AWS S3. This will allow us to:
- Create an S3 lifecycle policy to delete files after a set period of time
- Encrypt the files
- Provide signed download links back to the user, which will only allow the uploader to access the file
- Allow containers to scale if needed without breaking this functionality (files are currently tracked in-memory)
🏗️ Acceptance Criteria
- Application performs required checks prior to file upload
- Simulate S3 locally as part of the refiner's
docker-compose.yaml
setup (take a look at localstack) - Application should create a refined zip and store them in S3
- Upload uploading, the application should produce a signed S3 link that can be provided to the user for download
- We can likely remove the
/download
route fromdemo.py
since the returned download link will be a signed S3 link - Remove code from
demo.py
and elsewhere related to handling file cleanup
⚙️ Technical Notes
Localstack should spin up with the rest of the docker compose infrastructure. We may need some S3 related environment variables added to the application.
📜 Additional context
N/A