Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using S3 bucket to store request/response mapping templates instead of embedding in cloud formation template #589

Open
chandler-barlow opened this issue Apr 3, 2023 · 6 comments

Comments

@chandler-barlow
Copy link

I noticed that the vtl for response/request mapping templates is embedded in the final cloud formation json. How would I go about using ResponseMappingTemplateS3Location along with RequestMappingTemplateS3Location to use mapping templates stored in s3 rather than embedding in the cloud formation template itself?
It seems that this functionality is not currently supported by the plugin. Are there any plans to support it in the future? This would be a pretty useful feature.

Here is the docs referencing the option to host vtl in s3
Here is someone using this feature when writing the template by hand

@bboure
Copy link
Collaborator

bboure commented Apr 3, 2023

Thanks @chandler-barlow for the feedback.

Could you share your use-case?
Uploading resolvers to S3 vs "inline" is something I'd like to implement.
Are you talking about referencing an existing file that is not generated by the plugin?

@chandler-barlow
Copy link
Author

@bboure yes we are currently uploading our vtl to s3 as part of our deploy process and wanted to reference the upload mapping templates in our config. It would be awesome if the plugin could optionally upload the mapping templates and link them as part of deploying. Like let the user specify a specific s3 bucket to use and the plugin do the uploading/linking.

Thank you for the fast response! Really appreciate it

@ronnyroeller
Copy link

This would be very valuable indeed!

To share our use case: We've quite a lot of entities with complex VTLs (due to authorization logic). As the VTLs are inlined, we're running into the 1MB limit for the CloudFormation template file.

image

@bboure
Copy link
Collaborator

bboure commented Jul 5, 2023

Thank you all,

I have been discussing this recently with some teammates.
Here is the plan we have:

This plugin would upload the VTL/JS files to the deployment bucket (the same used for Lambda functions) and reference the paths to CloudFormation automatically. Ideally, the plan was to do this by default.

However, we hit a limitation that we are not sure how to solve yet. Uploading resolvers/mapping templates to S3 prior to the deployment phase would break substitutions. Substitutions rely on CloudFormation intrinsic functions in order to work (replace values), which does not allow us to support them with S3 uploads.

The alternative is to make S3 upload up-in and make clear that substitutions won't work.

@chandler-barlow
Copy link
Author

could you potentially resolve the substitutions at upload time? I wrote a hacky work around for my team that resolved the substitutions in advance instead of using the cloud formation substitutions. We were just using the substitutions like variables. I don't know if there is a more complex use for substitutions that I'm missing.
@bboure

@bboure
Copy link
Collaborator

bboure commented Jul 14, 2023

@chandler-barlow This is a solution I considered, but I don't think this is a good idea.
As you said, it's "hacky", and there are many things that people might want to substitute beyond the common use cases. It could literally be anything: S3 bucket name, RDS cluster name, Secret manager. variable, etc. Anything that you can deploy using CloudFormation. It's impossible to cover everything.

To that, add the complexity of dependencies. i.e. You can't resolve anything until the resource is created.

The real benefit of substitution is to use it with intrinsic functions. That is hard to emulate/resolve outside CloudFormation.

Possibly, one way to go around that would be to use a custom resource to upload the file.
This would highly increase complexity though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants