Skip to content

Add option to pass signed urls to workflow  #4

@cket

Description

@cket

Rather than download the redwood urls we should generate signed URLs and allow the pipeline to download them. This is necessary for autoscaling dockers.

To do this, I plan on modifying DockstoreRunner.download_and_transform_json to look at an argument along the lines of --generate_signed_urls. If that argument is passed, we can map the redwood url to an s3 url and use boto's generate_url function to create our signed url.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions