Contributing to Kubernetes is harder than it should be and problems have unclear owners

[BenTheElder]

Preserving this from a slack discussion in #sig-contribex as requested.

Will cleanup later. CC @cblecker @dims @nikhita

Pasted from slack:

---

IMHO a lot of them the problem is mostly "who owns this?"

• presubmits are awful ... -- bug testing? release? some other sig? who owns what tests run in presubmit? -- slow (> 1 hour long!) an inordinately flaky (can be 50+ /retest!) -- most of them can't be replicated locally without significant effort and $$ / access to cloud -- test output is often incomprehensible across the board, identifying actual causes is an art form

• staging can be very painful and confusing, especially if you've never seen it before ... -- unclear if there is any interest in ever removing that

• dev tools (hack,/, build/) are all thousands of lines of elaborate bash importing more elaborate and sometimes esoteric bash ... -- no sig really owns these? and the people that have OWNERS are all super busy

• we bury your PR in mountains of text / comment spam. -- testing? contribex?

• getting active approvers on your PR is something of an art form, we auto assign reviewers but many of them I've never seen review anything in a year or more -- who knows? contribex? testing?

... that's probably just a few of them, I don't really know who I'd bug to get any of those fixed because it's unclear who really owns them.

---

[beezerbt]

Thanks to @BenTheElder for this raising this issue. I want to write Golang code, and I am a huge fan of K8s. So I am really, i mean really motivated to contribute. Even if it means starting with documentation.

However the K8s landscape is bewildering. So I totally concur.

I mean whats the difference between kubernetes and kubernetes/community. A one line explanation would suffice in the README in either project. Maybe I miss something (hahaha) but there is no overall view of the codebase. Then there is the website...and so on.

I want to contribute, which is how I found myself here, but surely it must be easier than this. Non?

[BenTheElder]

I mean whats the difference between kubernetes and kubernetes/community.

FWIW the README in this repo does mention what it is for. https://github.com/kubernetes/community#kubernetes-community

Welcome to the Kubernetes community!

This is the starting point for joining and contributing to the Kubernetes community - improving docs, improving code, giving talks etc.

To learn more about the project structure and organization, please refer to Project Governance information.

Further down it goes on to explain more. This repo is for community docs etc., github.com/kubernetes/kubernetes contains much of the "main" project code. I suggest reading through some of the links in this repo's README to get started :sweat_smile:

However the K8s landscape is bewildering.

I want to contribute, which is how I found myself here, but surely it must be easier than this.

I think that's a separate problem that I'm not really sure what can be done about or where it would be tracked... Here we're tracking the difficulty in accomplishing a pull request to the main repo. I will think about how to clarify this issue, it was mostly filed to preserve a discussion about this problem in the community slack :sweat_smile:

[parispittman]

@castrojo and @nzouedi are going to help break this down into chunks for us! yay

[fejta-bot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

[nikhita]

/remove-lifecycle stale

[vishakhanihore]

/lifecycle frozen

[vishakhanihore]

Looking onto it @parispittman @nikhita !

[BenTheElder]

UPDATE:

• I've introduced a faster, less flaky, locally reproducible option for e2e presubmits (pull-kubernetes-e2e-kind and friends which are now in blocking presubmit) and we disabled flake attempts to start taking flaky tests more seriously ... but we still have the super slow tests and we currently have a major flake issue we hope may be resolved by the next runc release ...

• Staging is well, still tricky. not sure what to do there, one pattern being explored by others is moving code actually out to other repos, but this may not work for everything ...

• I plan to look at build issues / tools myself more later this year hopefully. I've also become an active approver of all of the build/hack scripts in the main kubernetes repo and we've almost completed a collective effort to get them ~passing shellcheck at least.

• Bot spam is mostly a matter of consensus which we can't seem to get. process / code changes would otherwise be pretty trivial. I don't know what else to do here. I've poked that bear a few times now :| See also: https://github.com/kubernetes/community/issues/3621

• re: active contributors to review / approve => https://github.com/kubernetes/community/issues/3179 is part of the puzzle, discussed today. One thought that came up is to require the SIGs to actively ACK that they've reviewed their OWNERS. Another is to use aliases more to limit the places we need to check. See also: https://github.com/kubernetes/community/issues/3947 https://github.com/kubernetes/community/issues/3945

ownership is still generally unclear.

[mrbobbytables]

/assign @castrojo

[BenTheElder]

/unassign @costrojo (jorge is doing other things now ...)

[BenTheElder]

some brain dump updates in the meantime:

• build stuff: I have a KEP to simplify the build more / expose developers to less of that. this will improve a bit in future releases.
• presubmit: SIG testing has worked with various SIGs / stakeholders to cut down on the stuff we run on PRs, particularly logn flaky jobs, but it's a constant battle. We're also working with the new WG reliability on efforts to track flakiness better and enforce fixing it, in addition to a pretty large overhaul of blocking presubmit + release jobs to run on new infra with guaranteed QOS for the pods, reducing flakiness
• bot spam is still locked up in approving any changes, we have the technology ...
• active approvers: well inactive members are getting pruned more actively now, but having active ones is still a problem. I have some test-infra issues related to the process being a bit broken, e.g. https://github.com/kubernetes/test-infra/issues/16793

[BenTheElder]

I've gotten https://github.com/kubernetes/enhancements/issues/2420 to alpha status now, meaning that contributors to the current Kubernetes development branch do not need to work with bazel AND make at least (just make).

ref comments like: https://twitter.com/tsaha/status/1366482303078207488

a small win at least.

re-raising https://github.com/kubernetes/community/issues/3621#issuecomment-791040517

---

/unassign @castrojo (mis-typed this above)

[BenTheElder]

see also: https://github.com/kubernetes/test-infra/issues/18729 related to reducing presubmit pain.

[sftim]

Can we split this into smaller, perhaps more approachable, issues?