Break out cyhy-nvdsync and cyhy-kevsync into AWS Lambdas

[mcdonnnj]

💡 Summary

The cyhy-nvdsync and cyhy-kevsync scripts in cisagov/cyhy-core should be broken out into Lambdas that are triggered with CloudWatch events on the same timer basis as the scripts are currently run in production.

Motivation and context

Given what the two scripts do:

• Retrieve data from a remote source.
• Minimally process the data.
• Add documents with this processed data to the appropriate collection.

it would make sense to break them out into Lambdas. This would align with other Lambdas in the CyHy system like cisagov/assessment-data-import and cisagov/findings-data-import. This would also remove the need to maintain an EC2 instance to run these tasks.

Implementation notes

Since cisagov/cyhy-core is written in Python 2 these two scripts would need to be ported to Python 3 and new ODMs would have to be written in a MongoDB ODM library that runs on Python 3 (since MongoKit does not support Python 3). Options that are currently maintained include:

• MongoEngine
• Ming
• Beanie (or Bunnet)

The projects would need to be based on cisagov/skeleton-aws-lambda-python to generate artifacts.

MongoDB consideration

This work must support MongoDB 3.6 as that is still what is currently in use.

Acceptance criteria

• cyhy-nvdsync is ported to a Lambda.
• cyhy-kevsync is ported to a Lambda.
• Appropriate Terraform is written in cisagov/cyhy_amis to provision these Lambdas and provide the necessary access to write to the database.