[rptest] Azure CDT bringup

[clee]

trafficstars

Bug fixes to tests and test infrastructure for running CDT on Azure. (Note: Currently includes commit from #18827, will rebase after that gets merged.)

Backports Required

• [x] none - not a (product) bug fix
• [ ] none - this is a backport
• [ ] none - issue does not exist in previous branches
• [ ] none - papercut/not impactful enough to backport
• [ ] v24.1.x
• [ ] v23.3.x
• [ ] v23.2.x

Release Notes

• none

Improvements

• replaces several instances of deprecated Cloud API usage with new Public API (because the old endpoints don't return Azure resources)
• successfully installs BYOC Azure agent
• allow override headers on POST requests for rpcloud_client
• workaround for /snap/bin not showing up in $PATH (making az CLI not show up as an available command)
• Azure support for config_profile_verify test

[CLAassistant]

All committers have signed the CLA.

[ivotron]

what I was trying to mention during our sync today: to ensure that these changes don't break HTT tests on other cloud providers, given that these are not exercised at PR time, some manual jobs can be triggered (pointing to this branch) so that they are executed.

[clee]

/cdt provider=aws

[clee]

/cdt provider=aws rp_version=build

[clee]

/cdt provider=aws rp_version=build

[clee]

/cdt provider=gcp rp_version=build

[vbotbuildovich]

'cdt_instance_type' and 'region' is required if 'provider' is not 'aws'

Workflow run logs.

[clee]

/cdt provider=gcp region=us-west-1 rp_version=build cdt_instance_type=n2-highmem-4

[vbotbuildovich]

new failures in https://buildkite.com/redpanda/redpanda/builds/51279#0190990e-2aff-46ac-ad35-6ae316956a3a:

"rptest.tests.services_self_test.KubectlLocalOnlyTest.test_is_redpanda_pod"

new failures in https://buildkite.com/redpanda/redpanda/builds/51287#0190999a-7a83-4c90-af52-5748d8545dd1:

"rptest.tests.services_self_test.KubectlLocalOnlyTest.test_is_redpanda_pod"

new failures in https://buildkite.com/redpanda/redpanda/builds/51287#019099a1-063a-4ac5-9ffa-fdcfea89efbe:

"rptest.tests.services_self_test.KubectlLocalOnlyTest.test_is_redpanda_pod"

new failures in https://buildkite.com/redpanda/redpanda/builds/51330#01909de2-a123-4152-a8e4-074c2331b304:

"rptest.tests.e2e_iam_role_test.AWSRoleFetchTests.test_write"

new failures in https://buildkite.com/redpanda/redpanda/builds/51338#01909f2a-f14c-4d26-a834-f865e353b0a5:

"rptest.tests.e2e_iam_role_test.ShortLivedCredentialsTests.test_short_lived_credentials"

new failures in https://buildkite.com/redpanda/redpanda/builds/51557#0190b970-9cb6-4a72-b795-0e40d6b949b7:

"rptest.tests.services_self_test.KubectlLocalOnlyTest.test_is_redpanda_pod"

new failures in https://buildkite.com/redpanda/redpanda/builds/51557#0190b971-ecbc-41f8-a486-57d9306b3d22:

"rptest.tests.services_self_test.KubectlLocalOnlyTest.test_is_redpanda_pod"

new failures in https://buildkite.com/redpanda/redpanda/builds/51557#0190bcdf-f17d-4ba1-91ad-4671061be6ec:

"rptest.tests.services_self_test.KubectlLocalOnlyTest.test_is_redpanda_pod"

new failures in https://buildkite.com/redpanda/redpanda/builds/51557#0190bcdf-ebfa-469d-965a-791b1607b6ce:

"rptest.tests.services_self_test.KubectlLocalOnlyTest.test_is_redpanda_pod"

new failures in https://buildkite.com/redpanda/redpanda/builds/51683#0190c24f-3171-4b90-bd13-b157d71c7bbc:

"rptest.tests.services_self_test.KubectlLocalOnlyTest.test_is_redpanda_pod"

new failures in https://buildkite.com/redpanda/redpanda/builds/51683#0190c267-ecc5-468a-8cba-9b845265128f:

"rptest.tests.services_self_test.KubectlLocalOnlyTest.test_is_redpanda_pod"

new failures in https://buildkite.com/redpanda/redpanda/builds/52018#0190e92b-7891-47d9-a4e0-fb75f653f972:

"rptest.tests.availability_test.AvailabilityTests.test_recovery_after_catastrophic_failure"

new failures in https://buildkite.com/redpanda/redpanda/builds/52018#0190ea59-4c39-4aa7-a84f-feaa9a9dae2e:

"rptest.tests.availability_test.AvailabilityTests.test_recovery_after_catastrophic_failure"

[clee]

/cdt provider=gcp region=us-west1 rp_version=build cdt_instance_type=n2-highmem-4

[vbotbuildovich]

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/51338#01909f15-275e-4af9-b361-20c1ed4c04e3

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/51557#0190b971-ecbc-41f8-a486-57d9306b3d22

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/51557#0190b971-ecbe-4968-9ead-22c220f423f9

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/51557#0190bcdf-f17d-4ba1-91ad-4671061be6ec

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/51557#0190bcdf-ebfa-469d-965a-791b1607b6ce

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/51683#0190c24f-3175-48d7-9b2b-7b699a372054

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/51683#0190c267-eccb-4fb5-ba15-2de26d525917

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/52018#0190ff3f-7af7-4bb8-9512-fc1de35fdc98

[clee]

/cdt provider=gcp region=us-west1 rp_version=build cdt_instance_type=n2-highmem-4 tests/rptest/test_suite_cloud.yml

[clee]

/cdt provider=gcp region=us-west1 rp_repo=nightly rp_version=latest cdt_instance_type=n2-highmem-4 tests/rptest/test_suite_quick.yml

[clee]

/cdt provider=aws rp_repo=nightly rp_version=latest tests/rptest/test_suite_quick.yml

[clee]

AWS CDT results:

================================================================================
| SESSION REPORT (ALL TESTS)
| ducktape version: 0.8.18
| session_id:       2024-07-24--001
| run time:         679 minutes 41.394 seconds
| tests run:        1590
| passed:           1464
| flaky:            0
| failed:           17
| ignored:          2
| opassed:          98
| ofailed:          9
| opassedfips:      0
| ofailedfips:      0
| ================================================================================

[clee]

/cdt provider=gcp region=us-west1 rp_repo=nightly rp_version=latest cdt_instance_type=n2-highmem-4 tests/rptest/test_suite_quick.yml

[clee]

GCP CDT results:

================================================================================
| SESSION REPORT (ALL TESTS)
| ducktape version: 0.8.18
| session_id:       2024-07-25--001
| run time:         707 minutes 4.610 seconds
| tests run:        1590
| passed:           1462
| flaky:            0
| failed:           19
| ignored:          2
| opassed:          98
| ofailed:          9
| opassedfips:      0
| ofailedfips:      0
| ================================================================================

[clee]

Pulling the trigger on this merge now. If any new CDT nightly failures show up, we'll deal with them as they come.