Need clarification on how and why Liqo generates AWS IAM users (declarative peering)

[jv4n5e]

It seems like the replicator controller is a black box that takes ResourceSlice CRs and generates Identity CRs, while taking the provided credentials to generate AWS IAM users. I am just assuming at this point since I could not find documentation that covers this design (or the rationale behind it), but it's an educated guess based on the CRDs.

In our environment, it is NOT okay to simply generate random AWS IAM users, especially if we can't control their naming convention. These liqo-* users don't even seem to have any particular policies or permissions, which is even more confusing in our use case (why not just stick with the provided credentials to do whatever it needs?)

Is it not possible to enable the ResourceSlice to take the authParams that the Identity CR should use, and if these are not provided, only then allow the Identity CR to generate random ones like it is doing already?
I have tried patching the Identity CR with predefined credentials, and I can see resources being reflected accordingly, but I would like to make sure this doesn't break anything else, or if it's only a temporary workaround that will break something in the long run.

Thanks!

[claudiolor]

Hi @jv4n5e,
the CR replicator replicates the resources on the provider cluster, so that requests can be forwarded from the consumer to the provider cluster: when you create a ResourceSlice, the CR replicator takes its specs and replicates them on the provider cluster, where the controller manger handles it. This latter component can accept or reject the request, and when the ResourceSlice is accepted, it creates the credentials that will be used by the virtual node. These credentials are then used by the controller manager of the consumer cluster to create the Identity resource.

Going to your question, the reason why you see multiple AWS IAM users, is that each virtual node has a corresponding user to enforce the resources quotas.

I agree having a random name for the generated user is not ideal, maybe something that can be done is defining a more talkative naming convention for the users (maybe something like liqo-$CONSUMER_CLUSTER_ID-$CONSUMER_PROVIDER_ID-$RS_NAME). However, we should consider that usernames can be 64 characters long and should not be unique.

Regarding your workaround, not sure whether the controller will enforce again the status, restoring the credentials it previously generated.

In any case you can still use the same credentials for everything: when you create a ResourceSlice you are delegating the provider cluster to generate the credentials for you. However, if you directly create a VirtualNode that refers to a secret with the kubeconfig with the permissions to do everything it needs, you can bypass the user creation and you can link all the VirtualNode resources to the same secret.

So, imagine you have a secret like the following:

apiVersion: v1
data:
  kubeconfig: <BASE64_KUBECONFIG>
kind: Secret
metadata:
  labels:
    liqo.io/identity-type: ControlPlane
    liqo.io/remote-cluster-id: <PROVIDER_CLUSTER_ID>
  annotations:
    liqo.io/remote-tenant-namespace: <PROVIDER_TENANT_NAMESPACE>
  name: cplane-secret
  namespace: <TENANT_NAMESPACE>

This is the secret used by the CR replicator to authenticate.
Instead of creating a ResourceSlice, you can create a VirtualNode referring to this secret:

apiVersion: offloading.liqo.io/v1beta1
kind: VirtualNode
metadata:
  labels:
    liqo.io/remote-cluster-id: <PROVIDER_CLUSTER_ID>
  name: my-virtual-node
  namespace: <TENANT_NAMESPACE>
spec:
  clusterID: <PROVIDER_CLUSTER_ID>
  createNode: true
  disableNetworkCheck: false
  kubeconfigSecretRef:
    name: cplane-secret
  labels:
    liqo.io/remote-cluster-id: <PROVIDER_CLUSTER_ID>
  resourceQuota:
    hard:
      cpu: "4"
      memory: 8Gi
      pods: 100
      OTHER_RESOURCES

In principle, the resulting user should have on the provider cluster the following cluster roles:

• liqo-remote-controlplane role for tenant namespace (to allow the operation done by the CR replicator),
• liqo-virtual-kubelet-remote for each of the offloaded namespace in the provider cluster (for the virtual nodes)
• liqo-virtual-kubelet-remote-clusterwide (for the virtual nodes).

If you have resourceEnforcement enabled on the provide cluster (controllerManager.config.enableResourceEnforcement=true), which is the default option, then you will need to create a quota on the provider side, where you specify the maximum amount of resources that the consumer is able to use (if you use the same credential, then the quota will be the aggregated consumed resources of all the virtual nodes):

apiVersion: offloading.liqo.io/v1beta1
kind: Quota
metadata:
  name: my-quota
  namespace: <PROVIDER_TENANT_NAMESPACE>
spec:
  resources:
    cpu: "4"
    memory: 8Gi
    pods: 100
    OTHER_RESOURCES
  user: <USERNAME>

[jv4n5e]

Thank you @claudiolor . This works as expected.
Wouldn't you guys consider updating the docs, providing an alternative for client authentication to use the "legacy" service account token secret? I ended up with this approach since your controllers don't currently support "aws exec" authentication, but it took some wall-hitting to get there (this may be subjective, as you do warn users about this, but I wanted to see for myself).