Add support for kubernetes version 1.30.0

[MichaelKatsoulis]

Proposed commit message

Add support to 1.30 Kubernetes version:

• Update documentation
• Change K8s cluster version to 1.30
• Update controller manager, scheduler and proxy metrics files.

Checklist

• [x] My code follows the style guidelines of this project
• [ ] I have commented my code, particularly in hard-to-understand areas
• [x] I have made corresponding changes to the documentation
• [x] I have made corresponding change to the default configuration files
• [ ] I have added tests that prove my fix is effective or that my feature works
• [ ] I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

• Relates to #3068

[botelastic[bot]]

This pull request doesn't have a Team:<team> label.

[mergify[bot]]

This pull request does not have a backport label. If this is a bug or security fix, could you label this PR @MichaelKatsoulis? 🙏. For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed branches, such as:

• backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

[elasticmachine]

:broken_heart: Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2024-04-25T13:57:30.775+0000

• Duration: 40 min 8 sec

Test stats :test_tube:

Test	Results
Failed	0
Passed	4583
Skipped	904
Total	5487

Steps errors

Expand to view the steps failures

Setup kind

• Took 0 min 13 sec . View more details here
• Description: .ci/scripts/kind-setup.sh

Integration tests

• Took 1 min 27 sec . View more details here
• Description: MODULE=kubernetes make -C metricbeat integration-tests

Error signal

• Took 0 min 0 sec . View more details here
• Description: Error 'hudson.AbortException: script returned exit code 2'

:green_heart: Flaky test report

Tests succeeded.

:robot: GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[MichaelKatsoulis]

Note:

Jenkins tests fail because it cannot create a kubernetes cluster with 1.30.0 version, as there is not such an image yet for kind in https://hub.docker.com/r/kindest/node/tags. We should wait until the image gets released.

[MichaelKatsoulis]

/test

[MichaelKatsoulis]

The PR is now ready for review! @fearful-symmetry or @faec could you also review it as part of data plane team?

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[tetianakravchenko]

Please update documentation https://github.com/elastic/beats/blob/main/metricbeat/module/kubernetes/_meta/docs.asciidoc#compatibility

[elasticmachine]

Pinging @elastic/obs-ds-hosted-services (Team:obs-ds-hosted-services)

[MichaelKatsoulis]

in those files is defined K8S_VERSION: https://github.com/elastic/beats/blob/main/.buildkite/deploy/kubernetes/deploy-k8s-pipeline.yml https://github.com/elastic/beats/blob/main/.buildkite/metricbeat/pipeline.yml https://github.com/elastic/beats/blob/main/.buildkite/filebeat/filebeat-pipeline.yml https://github.com/elastic/beats/blob/main/Jenkinsfile should it be also updated?

These tests are new. @oakrizan should we update those files, testing with the new kubernetes version as part of this PR ?

[MichaelKatsoulis]

This PR scope is to update and test the support of kubernetes version 1.30.0 for kubernetes metricbeat module . The tests and files of https://github.com/elastic/beats/pull/39210#pullrequestreview-2141720019 are not maintained by elastic/obs-ds-hosted-services team. So I will leave them out of this PR.