TargetGroupBindings can now manipulate target groups from different aws accounts

[marcosdiez]

Issue #3634

Description

Now, every time a TargetGroupBinding has the alb.ingress.kubernetes.io/IamRoleArnToAssume (and optionally alb.ingress.kubernetes.io/AssumeRoleExternalId) annotations, we assume it before interacting to it (actually, the assume role operation is cached).

In real life, I changed:

• targetgroupbinding_validator
• targetgroupbinding_mutator
• target manager (which has RegisterTargets / DeregisterTargets / ListTargets

This way we can easily manipulate target groups from target group bindings for every single AWS account in the world, as long as there is a role we can assume. In order to prevent the confused deputy problem, external id is supported.

The way this was implemented was actually way simpler than initially imagined:

the service.elbv2Client object now supports a new method called AssumeRole which returns service.elbv2Client on the specified role. The catch is that if AssumeRole is called with a blank role, the object returns itself and life goes on.

In other words, every time we call some method from elbv2Client from something related to target groups, we also call AssumeRole and that's basically it.

Caching the assumed role was also trivial, a simple map that is stored on cloud.go

I decided to have cloud.go manage the caching becuase in the future it could assume role for other objects if needed and all the assume role logic would be in the same place.

The only unfortunate change that needed to be made is that aws.Cloud became services.Cloud else golang would complain about circular dependency :(

This is what the logs look like:

{"level":"info","ts":"2024-05-13T03:24:31Z","msg":"awsCloud","method":"GetAssumedRoleELBV2","AssumeRoleArn":"arn:aws:iam::7550XXXXXXXX:role/alb-controller-policy-to-impersonate","externalId":"XXXXXX"}
{"level":"info","ts":"2024-05-13T03:24:33Z","msg":"registering endpoints using the targetGroup's vpcID vpc-00XXXXXXXXXXXXXXX which is different from the cluster's vpcID vpc-04XXXXXXXXXXXXXXX"}
{"level":"info","ts":"2024-05-13T03:24:33Z","msg":"registering targets","arn":"arn:aws:elasticloadbalancing:us-east-1:7550XXXXXXXX:targetgroup/helloworld-8080-a8054/5741da332459b560","targets":[{"AvailabilityZone":"all","Id":"10.244.0.22","Port":8080}]}
{"level":"info","ts":"2024-05-13T03:24:34Z","msg":"registered targets","arn":"arn:aws:elasticloadbalancing:us-east-1:7550XXXXXXXX:targetgroup/helloworld-8080-a8054/5741da332459b560"}

Documentation and tests were properly updated in this PR

Checklist

• Added tests that cover your change (if possible)
• Added/modified documentation as required (such as the README.md, or the docs directory)
• Manually tested
• Made sure the title of the PR is a good description that can go into the release notes

BONUS POINTS checklist: complete for good vibes and maybe prizes?! 🤯

• Backfilled missing tests for code in same general area 🎉
• Refactored something and made the world a better place 🌟

A working amd64 container with this code is available here: marcosdiez/aws-load-balancer-controller:v2.7.2-m2

update: on 2024-09-26 I rebased the code. The new container image is marcosdiez/aws-load-balancer-controller:v2.8.3-m2

update: on 2024-10-28 I rebased the code again :(. The new container image is marcosdiez/aws-load-balancer-controller:v2.9.0-m1

[k8s-ci-robot]

Hi @marcosdiez. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[marcosdiez]

@shraddhabang I believe my code is ready. Feel free to review at will! Thank you.

Also, if you don't want the CRDs to be changed, here:
126c9c8

they are untouched.

[shraddhabang]

Thank you @marcosdiez for this PR. This looks excellent.
I will take a look and test it out. And would also ask our team to review it. Please note that we will have to go through our formal security process to review this. We will add this to our next minor release v2.9.0.
Please feel free to reach out to us.

[shraddhabang]

/ok-to-test

[yu-croco]

Hi @shraddhabang ,

We will add this to our next minor release v2.9.0

Could you please share us when the v2.9.0 would release? We are looking forward to this feature.

[yu-croco]

Hi, @johngmyers @M00nF1sh @shraddhabang

We will add this to our next minor release v2.9.0.

Does anyone know the release schedule for v2.9.0 ?

[marcosdiez]

I just rebased and merged my code. There is also a container available here: marcosdiez/aws-load-balancer-controller:v2.8.3-m1

[marcosdiez]

Small update: the old docker container was broken. Please use marcosdiez/aws-load-balancer-controller:v2.8.3-m2 instead (it was not taking ExternalID into consideration). The code has been rebased and updated again.

[marcosdiez]

/retest

[chkp-benzy]

Hey @marcosdiez, 10x for the great work

i tried your image (marcosdiez/aws-load-balancer-controller:v2.8.3-m2) and it works great :)
the targets were synced to the remote account ALB targetgroup :)

FYI, at first i didn't provided the necessary permissions and the pod crashed, with a good error message :)
i think that the error message is enough, it will be great if the pod wont crash :)

waiting for a Merge to main :)

[marcosdiez]

@shraddhabang any plans on merging this ? It took me 8 hours for this last rebase....

[roliverio]

@shraddhabang appreciate if you can speed up this merge/release. There's significant work done on it now.

[yardenws]

Awesome!
Waited for this for a very long time!

Please merge it ASAP🙏🏻

[abiloski]

This is great!!
Can you please approve and merge it 🙏🏻

[listellm]

I am so desperate for this feature. I get excited when there is a new release but then see that it is not included 😢

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: marcosdiez
Once this PR has been reviewed and has the lgtm label, please assign m00nf1sh for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment