allow external hpa

[robinvd]

Motivation

We need automatic scaling for servingruntimes based on some gpu load metric. The default deployment included metrics including prometheus. Using keda we want to scale based on prometheus queries.

The current options for scaling are

• static based on replicas in the servingruntime/config
• dynamic using hpa annotation. But this creates a managed hpa that only works with the buildin kubernetes metrics. Adding another hpa will conflict with the already created one.

This PR adds the external option where the controller wont set the replicas and wont create an hpa.

Modifications

The already existing External AutoscalerClass is now checked for instead of crashing the controller. The behavior is the same as None except it does not set the replicas property.

Result

potentially/partially solves https://github.com/kserve/modelmesh-serving/issues/372

[oss-prow-bot[bot]]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: robinvd Once this PR has been reviewed and has the lgtm label, please assign rafvasq for approval by writing /assign @rafvasq in a comment. For more information see:The Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

[oss-prow-bot[bot]]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: robinvd Once this PR has been reviewed and has the lgtm label, please assign rafvasq for approval by writing /assign @rafvasq in a comment. For more information see:The Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

[vinismarques]

Hey @robinvd, have you been using HPA based on GPU load successfully? I would love to see this being integrated.

[robinvd]

@vinismarques yes using this setup im running an autoscaling setup based on gpu load.