Allow autoscale based on custom values? #551
Description
The autoscale now looks for target requests only to enable scaling.
Can it use other metrics like gpu utlisation, kv cache etc??
{{ message }}
The autoscale now looks for target requests only to enable scaling.
Can it use other metrics like gpu utlisation, kv cache etc??