[WIP] Skip minScale during rollout for specific workloads

[skonto]

Proposed Changes

• When in roll out, skip minScale until traffic shifts for workloads with a specific annotation features.knative.dev/skip-min-scale-during-rollout: "Enabled". Although we dont provide guarantees for a smooth traffic transition this can help with certain use cases where expensive resources are used e.g. gpu and we don't optimize for latency e.g. batch or when there is latency tolerance in general. We rely on the activator to manage pending requests (backpressure). Although does not completely fix https://github.com/knative/serving/issues/12971 it unblocks some scenarios until the functionality of the progressive rollout repo is ready. Also it can be useful when utilized during low traffic hours in production. Btw I created this PR to discuss the idea in general.

• Old pods still do suffer from the terminationGracePeriod (linked to the revision timeout) but at least we can go from 2x resources needed in the worst case to x+N. Maybe we could also relax minScale for the old revision (rely on the autoscaler to scale it before traffic shift is done).

• Here is an example where: a) we start with minScale=2 and serving.knative.dev/rollout-duration: "380s" so we have enough time to observe. b) we update the minScale to 210 (random scale) and wait until traffic shifts, and we enter a roll-out. Note here that normally the new revision will serve traffic but will never be active without this patch, eventually progressdeadline will expire if there are no resources. c) While traffic is shifting the new revision will skip minScale and autoscale based on available resources and max scale. d) Later on since there is no traffic, both revisions have no replicas (the initial revision is scaled down too) and we have exited the roll-out. e) When a request comes in the minScale is respected again for the latest revision, since we exited the roll-out, and we operate as usual.

NAME                  CONFIG NAME     GENERATION   READY   REASON   ACTUAL REPLICAS   DESIRED REPLICAS
helloworld-go-00001   helloworld-go   1            True             2                 2

NAME                  CONFIG NAME     GENERATION   READY     REASON   ACTUAL REPLICAS   DESIRED REPLICAS
helloworld-go-00001   helloworld-go   1            True               2                 2
helloworld-go-00002   helloworld-go   2            Unknown            0                 210

NAME                  CONFIG NAME     GENERATION   READY     REASON   ACTUAL REPLICAS   DESIRED REPLICAS
helloworld-go-00001   helloworld-go   1            True               2                 2
helloworld-go-00002   helloworld-go   2            Unknown            0                 210

NAME                  CONFIG NAME     GENERATION   READY   REASON   ACTUAL REPLICAS   DESIRED REPLICAS
helloworld-go-00001   helloworld-go   1            True             2                 2
helloworld-go-00002   helloworld-go   2            True             3                 1

NAME                  CONFIG NAME     GENERATION   READY   REASON   ACTUAL REPLICAS   DESIRED REPLICAS
helloworld-go-00001   helloworld-go   1            True             2                 2
helloworld-go-00002   helloworld-go   2            True             0                 0

NAME                  CONFIG NAME     GENERATION   READY   REASON   ACTUAL REPLICAS   DESIRED REPLICAS
helloworld-go-00001   helloworld-go   1            True             0                 0
helloworld-go-00002   helloworld-go   2            True             0                 0

NAME                  CONFIG NAME     GENERATION   READY   REASON   ACTUAL REPLICAS   DESIRED REPLICAS
helloworld-go-00001   helloworld-go   1            True             0                 0
helloworld-go-00002   helloworld-go   2            True             0                 210

NAME                  CONFIG NAME     GENERATION   READY   REASON   ACTUAL REPLICAS   DESIRED REPLICAS
helloworld-go-00001   helloworld-go   1            True             0                 0
helloworld-go-00002   helloworld-go   2            True             1                 210

Release Note

Allow to skip minScale during in roll-out for specific workloads.

[knative-prow[bot]]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: skonto Once this PR has been reviewed and has the lgtm label, please assign dprotaso for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• ~~OWNERS~~ [skonto]
• pkg/apis/OWNERS

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 51.61290% with 30 lines in your changes missing coverage. Please review.

Project coverage is 80.68%. Comparing base (3f7fd94) to head (fd476c1). Report is 102 commits behind head on main.

Files with missing lines	Patch %	Lines
pkg/reconciler/autoscaling/kpa/kpa.go	55.81%	16 Missing and 3 partials :warning:
pkg/reconciler/autoscaling/kpa/scaler.go	35.29%	10 Missing and 1 partial :warning:

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #15780      +/-   ##
==========================================
- Coverage   80.81%   80.68%   -0.14%     
==========================================
  Files         222      222              
  Lines       18075    18114      +39     
==========================================
+ Hits        14608    14615       +7     
- Misses       3096     3124      +28     
- Partials      371      375       +4     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[skonto]

cc @houshengbo this might interest you, pls let me know what you think.

[github-actions[bot]]

This Pull Request is stale because it has been open for 90 days with no activity. It will automatically close after 30 more days of inactivity. Reopen with /reopen. Mark as fresh by adding the comment /remove-lifecycle stale.