Batch support for Route53 module

[danmoz]

Summary

Modify the route53 module to send multiple record updates in a batch. This would dramatically speed up playbooks which need to update a large number of records.

Issue Type

Feature Idea

Component Name

amazon.aws.route53

Additional Information

I find that the slowest parts of my playbooks are always those that update Route53 DNS records. By using asynchronous actions, I was able to speed things up a little, but it's still quite painful for bulk updates.

I noticed the Route53 API accepts batched requests in the form of ChangeRecordSets; if the module took advantage of this feature, updating 20 records would take about the same amount of time as updating a single record (i.e. O(n) to O(1) time).

This would be a game changer for my use case and I suspect many others. It would also reduce the risk of being rate limited by AWS.

Code of Conduct

• I agree to follow the Ansible Code of Conduct

[markuman]

@felixfontein If I remember right, you wanted to implement such feature long time ago :)

@danmoz my workaround for this is to keep all your zone records in a var file (or roles default main.yml) - let's call it requested.
First I use route53_info to see what exists (register: EXISTS). 2nd I compare what exists with what is requested. What's left are records that are changed or new.

    - name: records that need to be created or updated
      set_fact:
        ADD_AND_UPDATE: "{{ REQUESTED | difference(EXISTS.ResourceRecordSets) | list }}"

Now use route53 module to loop over ADD_AND_UPDATE for all records that need to be updated or are completely new.

Next, re-read what exists now using route53_info (register: EXISTS) and flip the comparsion. What's left here are records that don't exists anymore and must be deleted.

    - name: records that need to be removed
      set_fact:
        REMOVE: "{{ EXISTS.ResourceRecordSets | difference(REQUESTED) | list }}"

Loop again with REMOVE over route53 module to remove records that don't exists in your var file.

The whole thing assumes that you must put all your zone record once in a var file.
It reduces here our rollout from >20 minutes to <1 minute.

[danmoz]

That's a really nice workaround @markuman! Thanks for sharing.

I'd still love to have batch support in Ansible though 😉

[felixfontein]

@felixfontein If I remember right, you wanted to implement such feature long time ago :)

Definitely! Never got around to do it unfortunately...