Open
Description
💡 Summary
Currently, a few of the PE Report scans utilize SQS parallelization for increased performance. We need to apply this SQS parallelization to the rest of the scans as well.
Motivation and context
We may need to scale our operations in the near future. As a result, we need to prepare our code to be able to generate increasingly large numbers of reports and not take forever to execute.
Implementation notes
- Scans need to be reconfigured to be able to run on a specified list orgs
- Scans need to be easily callable via command line
- Scans need to be able to support a dynamic number of available API keys
Acceptance criteria
- Identify which scans need to be updated for SQS
- Implement SQS versions of scans
- Test SQS scans to ensure everything is still working properly
- Stress test SQS scans to ensure no APIs/infrastructure is being overloaded