Feature Request: Add ability to scale multiple services in a group

[jaxtonw]

Describe the feature

Having the ability to scale multiple Knative Services in a group would be extremely beneficial. A single Knative Eventing Sequence may have multiple services involved. If requests go to the first step of the sequence, it would reduce latency to allow the entire sequence to scale in unison based on metrics from the request to the first sequence step. Outside of Knative Eventing sequences, an API may utilize multiple Knative Services to complete a request. Based on the request load coming into the first service (or any other arbitrary service), it would be beneficial to note that other services should scale up in anticipation of this increased load.

This would come in handy particularly when scaling many services up from 0. While many techniques can be employed to get Knative Services to start up quite fast, one of the current bottlenecks is that multiple services do not scale until they are needed. If multiple services are needed in sequence, the time to scale up time becomes summative and can quickly go beyond acceptable levels. Requests do not complete until all services scale up and complete in order.

Being able to define a "scaling group" would enable users to define a group of services that should have certain autoscaling metrics considered in aggregate. Load coming into service A (the leader of the scaling group) can be used as an indicator for anticipated load to others services B, C, etc. enabling these other services to scale in anticipation of this load. Instead of scale up times being summative, all scaling happens simultaneously, significantly reducing the time it takes for a sequence of services to complete a task during autoscaling operations, especially when scaling from 0.

[jaxtonw]

I've intentionally placed this feature request on Knative Serving instead of Knative Eventing as it pertains directly to Knative Services. While it would significantly impact the Knative Eventing project (ie: scaling groups based on Knative Eventing Sequences/Parallels), it is a standalone feature request that could extend beyond the Knative Eventing use cases.

[dprotaso]

FYI there was a past attempt at this you can see the issue here: https://github.com/knative/serving/issues/13145 But they never got past a proposal stage

One proposed building a graph automatically with tracing, the other was more of a simple CRD that mapped various components together

What kind of cold start times are you encountering?

[github-actions[bot]]

This issue is stale because it has been open for 90 days with no activity. It will automatically close after 30 more days of inactivity. Reopen the issue with /reopen. Mark the issue as fresh by adding the comment /remove-lifecycle stale.