cluster-api icon indicating copy to clipboard operation
cluster-api copied to clipboard

Consolidate dualstack and e2e-full test jobs

Open killianmuldoon opened this issue 2 years ago • 12 comments

Cluster API currently has two large e2e test runs - on main these are called [capi-e2e-main](https://github.com/kubernetes/test-infra/blob/b32aa0f189f4bc543753ce33f99ee88068dbfdaf/config/jobs/kubernetes-sigs/cluster-api/cluster-api-periodics-main.yaml#L70-L111) and [capi-e2e-dualstack-and-ipv6-main](https://github.com/kubernetes/test-infra/blob/b32aa0f189f4bc543753ce33f99ee88068dbfdaf/config/jobs/kubernetes-sigs/cluster-api/cluster-api-periodics-main.yaml#L112-L155C20).

capi-e2e-dualstack-and-ipv6-main runs everything from capi-e2e-main and additionally runs the CAPI quickstart with IPv6 as well as quick-start with dualstack tests for dualstack IPv4 primary and dualstack IPv6 primary.

We should consider de-duplicating these tests by running the current capi-e2e-dualstack-and-ipv6-main config under capi-e2e-main and removing the capi-e2e-dualstack-and-ipv6-main periodic and presubmit jobs.

Note: This can not be done before resolving https://github.com/kubernetes-sigs/cluster-api/issues/8816. That issue concerns flakes in the dualstack tests. These flakes signal that something is likely not right in our dualstack configuration.

killianmuldoon avatar Nov 03 '23 11:11 killianmuldoon

/triage accepted

killianmuldoon avatar Nov 03 '23 11:11 killianmuldoon

/kind cleanup /priority backlog

fabriziopandini avatar Apr 11 '24 16:04 fabriziopandini

@chrischdi @killianmuldoon After #8816 has been fixed. Should we consider this issue again?

sbueringer avatar Apr 15 '24 14:04 sbueringer

IIRC this would basically mean we'll use the dualstack job config for e2e-main and delete the dualstack job

sbueringer avatar Apr 15 '24 14:04 sbueringer

Yeah IMO we should go ahead and do this

killianmuldoon avatar Apr 15 '24 14:04 killianmuldoon

/assign

killianmuldoon avatar Apr 15 '24 14:04 killianmuldoon

Just to surface this here as well. #10135 is merged, we should be now unblocked to continue this issue

sbueringer avatar Apr 18 '24 08:04 sbueringer

Started taking a look at this - do we want to remove the seperate ipv6 jobs on presubmits? Do we want to make the change for all branches? i.e. 1.5, 1.6, 1.7, main.

Getting used to the prowjob generator - great work on that @chrischdi

killianmuldoon avatar Apr 30 '24 11:04 killianmuldoon

I would merge the ipv6 and the regular job, which IIRC basically means keeping the ipv6 job with the name of the other job.

I don't remember the exact fixes and how far we backported them. Ideally we would merge the jobs for all branches that are now stable (@chrischdi can probably provide more info on that)

sbueringer avatar Apr 30 '24 11:04 sbueringer

The fix got cherry-picked back to 1.5, so should be on all relevant branches.

I think we can use k8s-triage to compare two jobs we want to merge, to see if there errors are ~ the same or different.

Or used the combined query: https://storage.googleapis.com/k8s-triage/index.html?job=.-cluster-api-e2e-dualstack-and-ipv6-main%7C.-cluster-api-e2e-main

chrischdi avatar Apr 30 '24 12:04 chrischdi

https://github.com/kubernetes/test-infra/pull/32545

Should be ready for review now

killianmuldoon avatar May 08 '24 08:05 killianmuldoon

Just want to want to emphasize. Thx @killianmuldoon @chrischdi for all the work on initially making IPv6 work until now getting rid of the separate IPv6 jobs!

sbueringer avatar May 08 '24 17:05 sbueringer

@killianmuldoon @chrischdi I think this is done

sbueringer avatar Jul 10 '24 04:07 sbueringer

Yeah this is done - closing.

killianmuldoon avatar Jul 10 '24 07:07 killianmuldoon