[FLINK-34947] Only scale down JM in Foreground deletion propagation and reduce timeout

[gyfora]

What is the purpose of the change

We introduced a logic to scale down the JobManager before the task managers are killed to have a more graceful shutdown sequence.

Currently this is always done in native mode, but it makes more sense to restrict this functionality to foreground deletion. Foreground deletion is the default and recommended propagation but in some cases users may want to use background deletion to speed up the process. In these cases we should not force a synchronous, potentially slow step.

Furthermore instead of using the entire deletion timeout we should reduce this to only use a portion of it as it's an optional step and we should always have keep enough time for the TM shutdown.

Brief change log

• Only shut down JMs first in Foreground deletion
• Use only half the total shutdown timeout for JM shutdown but not more than 1 minute
• Comments/method name improvements
• Unit tests

Verifying this change

New unit tests have been added to verify the exact behaviour

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changes to the CustomResourceDescriptors: no
• Core observer or reconciler logic that is regularly executed: yes

Documentation

• Does this pull request introduce a new feature? no
• If yes, how is the feature documented? not applicable