🤔 Agent Pool Geo-Redundancy

[fbinotto]

Operator Version, Kind and Kubernetes Version

• Operator version:
• Kind: Workspace | Module
• Kubernetes version:

YAML Manifest File

# Copy-paste your YAML manifest here

Output Log

Kubectl Outputs

Question

References

Community Note

• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.

[fbinotto]

Hum, not sure what happened with my question. Repeating it here.

Can I have 2 AKS clusters with the same configuration in 2 separate regions, managing the same Agent Pool?

[fbinotto]

Anyone? @arybolovlev

[arybolovlev]

Hi @fbinotto,

There will be several issues if two Agent Pool controllers manage the same agent pool, as there is no synchronization mechanism between them:

1. Token Management – since states are not synchronized, each controller will remove tokens created by the other and generate its own. This will result in tokens having a very short lifetime, likely preventing agents from communicating with HCP Terraform. In theory, this could be mitigated through some form of manual or automated status synchronization, but I wouldn't recommend this approach, as it may lead to unpredictable behavior.

2. Agent Management – both controllers might detect the need to scale up simultaneously and start provisioning agents. However, only one of these agents will ultimately pick up a pending run. This could lead to overprovisioning in some cases.

We are exploring the development of a new controller that will monitor Agent Pool and expose the scale subresource, enabling HPA or KEDA to manage agent scaling. Since this is still in the discussion phase and we are gathering use cases, I can’t share many details at this time.

It would be great if you could share more details about your requirements and setup so we can design the new controller more thoughtfully. If you are not comfortable sharing this information here, feel free to reach out to us via email at hcp-tf-operator@hashicorp.com.

Thank you!

[fbinotto]

Thanks for the reply @arybolovlev

We are using the AKS Cluster solely to manage a single Agent Pool. We have around 6000 workspaces attached to this Agent Pool.

My concern is if the cluster goes down, we would not have any available agents. Are there any options for this scenario, other than switching back to managed agents or having a secondary Agent Pool linked to another AKS cluster and if something goes wrong with the primary, manually switch all the workspaces to the secondary?

Can a workspace be linked to more than 1 Agent Pool? I don't think it can.

Thanks,
Felipe

[fbinotto]

Good day @arybolovlev

Any feedback on the question above?

Thanks,
Felipe

[fbinotto]

@arybolovlev