flakes in clusterctl upgrade tests

[cahillsf]

summarized by @chrischdi 🙇

---

According to aggregated failures of the last two weeks, we still have some flakyness on our clusterctl upgrade tests.

• 3 Failures: Internal error occurred: failed calling webhook [...] connect: connection refused

  • Component: CAPD
  • Branches:
    • main
    • release-1.8

• 2 Failures: x509: certificate signed by unknown authority

  • Component: unknown
  • Branches:
    • main
    • release-1.8
    • release-1.7

• 2 failures: failed to run clusterctl version:

  • Component: clusterctl/test framework it seems
  • Branches:
    • release-1.7
    • release-1.6

• ~~5 Failures~~: Timed out waiting for Machine Deployment clusterctl-upgrade/clusterctl-upgrade-workload-... to have 2 replicas

  • [2024-10-21 update] no failures in recent history but did have some as recently as 2024-10-03. Unclear what introduced the improvement (this was relatively infrequent to begin with)
  • Component: unknown
  • Branches:
    • release-1.8
    • main

• ~~2 Failures: Timed out waiting for Cluster clusterctl-upgrade/clusterctl-upgrade-workload-... to provision~~ - resolved with: https://github.com/kubernetes-sigs/cluster-api/pull/11220

  • Component: unknown
  • Branches:
    • release-1.8
    • main

• ~~36 failures: Timed out waiting for all Machines to exist~~ split off into: https://github.com/kubernetes-sigs/cluster-api/issues/11209

• ~~16 Failures: Failed to create kind cluster~~ - resolved with: https://github.com/kubernetes-sigs/cluster-api/pull/11220

  • Component: e2e setup
  • Branches:
    • main
    • release-1.7

Link to check if messages changed or we have new flakes on clusterctl upgrade tests: here

/kind flake