flink-kubernetes-operator
flink-kubernetes-operator copied to clipboard
[FLINK-36527][autoscaler] Introduce a parameter to support autoscaler adopt a more radical strategy when source vertex or upstream shuffle is keyBy
What is the purpose of the change
Introduce a parameter to support autoscaler adopt a more radical strategy when source vertex or upstream shuffle is keyBy
Brief change log
-
Add a new option:
scaling.key-group.partitions.adjust.mode
- Support use a more aggressive strategy(Resource utilization takes priority) to determine the degree of parallelism after Source or after keyby without first considering balanced consumption.
Verifying this change
in org.apache.flink.autoscaler.JobVertexScalerTest.JobVertexScalerTest
-
testParallelismComputationWithAdjustment
andtestNumPartitionsAdjustment
add logic to test
Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (yes / no)
- The public API, i.e., is any changes to the
CustomResourceDescriptors
: (yes / no) - Core observer or reconciler logic that is regularly executed: (yes / no)
Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)