etcd icon indicating copy to clipboard operation
etcd copied to clipboard

Focus on fixing flaky test cases and bugs

Open ahrtr opened this issue 1 year ago • 3 comments
trafficstars

What would you like to be added?

There are 43 open bugs (label: type/bug) and 11 flaky test issues (label: type/flake) at the moment of writing. We also see flaky workflows from time to time, usually the PR authors just ignore the failures as long as the failures have nothing to do with their PRs.

I'd suggest the community to focus on fixing flaky test cases, bugs and failed workflows (we may not create issue for each failed workflow) , and on-hold development on new features until we feel comfortable (e.g. flaky test issue + bugs < 10).

Why is this needed?

To improve stability and gain more confidence.

cc @Elbehery @jmhbnz @fuweid @serathius @spzala

ahrtr avatar Jan 25 '24 10:01 ahrtr

thanks @ahrtr

I am actually assigned to https://github.com/etcd-io/etcd/issues/13919

it targets the e2e tests .. I tried going through the test framework to find my way, but i was lacking two main things

  • test framework design / docs
  • specific requirements for https://github.com/etcd-io/etcd/issues/13919

also maybe we can triage some for todays meeting @jmhbnz

Elbehery avatar Jan 25 '24 11:01 Elbehery

SGTM. Will keep more eyes on this.

fuweid avatar Jan 25 '24 14:01 fuweid

Based on https://github.com/etcd-io/etcd/actions/runs/7597605377/job/20692607792 we see 13% of flake. We should identify the source of flake.

serathius avatar Jan 25 '24 19:01 serathius

Discussed during sig-etcd triage meeting. @ahrtr did you have any specific action items for the community to complete for this? The issue is generic so are there sub issues underneath?

jmhbnz avatar Jun 06 '24 18:06 jmhbnz