feature: add scalePolicy in lws

[googs1025]

What would you like to be added:
I want to implement a function similar to scalePolicy, the api is as follows.

apiVersion: leaderworkerset.x-k8s.io/v1
kind: LeaderWorkerSet
metadata:
  name: leaderworkerset-sample
spec:
  scalePolicy:
    minReplicas: 1
    maxReplicas: 3
    metrics:
      - type: Resource
        resource:
          name: cpu
          target:
            type: Utilization
            averageUtilization: 80
  replicas: 3
  leaderWorkerTemplate:
    size: 4
    workerTemplate:
      spec:
        containers:
        - name: nginx
          image: nginx:1.14.2
          resources:
            limits:
              cpu: "100m"
            requests:
              cpu: "50m"
          ports:
          - containerPort: 8080

We can implement the scale subresource (it seems to have been implemented now) and embed HPA, so that users can use scalePolicy directly without using HPA separately.

This would involve an api-change, I'll contribute a KEP if needed.

Why is this needed:

• users can use scalePolicy directly without using HPA separately.

Completion requirements:

This enhancement requires the following artifacts:

• Design doc
• API change
• Docs update

The artifacts should be linked in subsequent comments.

[googs1025]

/assign

[googs1025]

/kind api-change

[googs1025]

@ahg-g @liurupeng @kerthcet Any ideas or suggestions?

[kerthcet]

But how to scale, aren't we also depend on the HPA implementation. If so, I don't think this is fit with lws as an plain workload. Some higher orchestration tools may need this for easy configuration, if you're interest, you can take a look at InftyAI/llmaz#3, we're aiming for the simplified configuration.

[ahg-g]

Right, why do we need this if we have HPA?

[googs1025]

thanks！

My thought is to integrate HPA through the API in LWS (just like we use a headless service), so that users wouldn't need to configure HPA separately and could manage it directly from within LWS.

[ahg-g]

The headless service case is different, it is an implementation detail of enabling pod to pod communication, it is not easy to setup independently because in some cases they need to be created per replica. Pod to pod communication is important for LWS because it aims to handle distributed inference.

LWS is like Deployment, they are API to deploy the replicas of a workload. Autoscaling and load balancing are intentionally not baked into deployment apis to enable composability.

Autoscaling and load balancing are features that can be setup in different ways, autoscaling can be done using HPA or KEDA; load balancing can be setup using Service or Gateway API. The deployment api (Deployment or LWS) should not impose a specific way of doing that, and hence it is not in their scope.

[googs1025]

Thank you all for your suggestions. I will close this issue.：）