Build a consumer for AWS SNS streams

[troyraen]

The AWS Simple Notification Service (SNS) is similar to Google Pub/Sub. It would be great for us to build a consumer that can ingest an SNS topic into Pub/Sub.

How?

SNS can deliver messages to an HTTPS endpoint, so we ought to be able to write a Cloud Run module that will do this. See Subscribing an HTTP/S endpoint to an Amazon SNS topic. The process appears to be very similar to connecting Cloud Run to Pub/Sub. Basically:

• Write a Cloud Run module that accepts an incoming HTTP request and publishes the data and metadata to Pub/Sub.
• Deploy to GCP.
• GCP will generate an endpoint URL for the deployed module. Use it to create a subscription to the SNS topic.
• SNS notifications in that topic should then be automatically delivered to the deployed module.

Possible astronomy "alert" streams

I am aware of SNS topics from both HST (Hubble Space Telescope) and TESS (Transiting Exoplanet Survey Satellite) that could be interesting. Both are provided by the MAST archive through the AWS Open Data Repository (ODR). I haven't been able to find much info about what is actually in those streams, but the following pages have high-level descriptions and connection info:

• MAST Data Collections On The Cloud
• TESS ODR page
• HST ODR page