[FLINK-33914][ci] Adds basic Flink CI workflow

[XComp]

Based on the following PR(s):

• https://github.com/apache/flink/pull/23961
• https://github.com/apache/flink/pull/23962
• https://github.com/apache/flink/pull/23965
• https://github.com/apache/flink/pull/24061
• https://github.com/apache/flink/pull/23964
• === THIS PR ===
• https://github.com/apache/flink/pull/23971
• https://github.com/apache/flink/pull/23972

What is the purpose of the change

Creates CI workflow that runs on Flink's default setup in branches.

Brief change log

• Adds custom actions for selecting the Java version
• Adds custom action for moving data within the container
• Adds template for pre-compile checks
• Adds template for executing Flink's test suite
• Adds basic CI workflow

Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changed class annotated with @Public(Evolving): no
• The serializers: no
• The runtime per-record code paths (performance sensitive): no
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
• The S3 file system connector: no

Documentation

• Does this pull request introduce a new feature? no
• If yes, how is the feature documented? not applicable

[flinkbot]

CI report:

• 91c958893a7c1744761242bb777f4583cd053900 UNKNOWN
• afeca9b46ea14bd3ba9576efb9e50f151f343317 UNKNOWN
• 3bdf3c458244236f349d6ba871e4bdbc1eaf1d26 Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[XComp]

~I reorganized the PRs. This one is now including FLINK-34047 and the updated FLINK-33906. This workflow run includes all the changes and should fail because I haven't set the parameters that are need for running tools/azure-pipelines/uploading_watchdog.sh (see corresponding code snippet in PR #24061)~

:facepalm: I forgot that I should add the env variables as a step and not within the GitHub environment. I will do the change now.

[XComp]

Ok, looks like the CI pipeline is working again: https://github.com/XComp/flink/actions/runs/7475753305/job/20344970380 (the failure in packaging/licensing happened 32019a6f6d7371c0fce23978086324f9583f9534 seems to have been lost in one of the rebasings. I cherry-picked it into the PR again (GHA workflow).

[XComp]

I triggered another workflow with lower timeouts: https://github.com/XComp/flink/actions/runs/7478072394 ...to double-check that running into time outs works as expected.

[XComp]

Related GHA workflow run

[XComp]

Force push change removed the inputs.branch parameter which is not needed anymore when you use branch-specific configuration (related GHA workflow run)

[XComp]

@flinkbot run azure

[XComp]

The force push above is a plain rebase to most recent master to include FLINK-33907 and FLINK-34150

[XComp]

The past force push squashes the following changes into the ready made PR:

• Fixing the ubuntu version to 22.04 instead of relying on the latest
• Changing CI image to a new version that is based on Ubuntu 22.04 with OpenSSL installed
• Added older OpenSSL version to e2e test step (that wasn't causing any issues in the past because of FLINK-34150)
  • Here's a nightly workflow run and a CI workflow run without this fix
• Update of the *-artifact actions from v3 to v4 (more details in FLINK-33923)

[XComp]

The past two force pushes add the following changes:

• --privileged parameter was added
• container image was changed to the result of FLINK-34194

[XComp]

• This force push updates the workflow name to include (beta)
• The subsequent force push adds functionality to resetting DEBUG_FILES_OUTPUT_DIR. This disables the warning in the AdaptiveScheduler stage (see example run w/o the fix and another run w/ the fix)

[XComp]

Deprecation warning started to appear which forced me to update the version for actions/checkout and actions/cache to v4.