Documentation for the conformance test suite

[shaneutt]

It's difficult for a newcomer to our conformance tests today to figure out how to get started running the conformance test suite against their implementation. It can be unclear that there are both CLI and Go library ways to do this, and what "profiles" are and how conformance test results work.

The purpose of this issue is to add documentation, with helpful links from several places to help someone get from "I have an implementation and have no idea how to run conformance" to "I am running conformance regularly in my CI, and I'm submitting ConformanceReports back to upstream".

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[shaneutt]

This is a requirement for standard graduation of the KEP, and will be worked on soon, not stale.

/remove-lifecycle stale

[shaneutt]

Was talking with @mlavacca and he's gonna work on this last little bit!

/assign @mlavacca

Thanks Mattia, we don't need anything huge, just a conformance/README.md which includes a section that describes how to use the conformance test suite for new implementations.

[shaneutt]

Also, now that this is the last thing we need to do in order to consider #1709 Standard, note that the PR which resolves this can also have the honors of updating the GEP to Standard! :partying_face:

[mlavacca]

Also, now that this is the last thing we need to do in order to consider #1709 Standard, note that the PR which resolves this can also have the honors of updating the GEP to Standard! 🥳

🎉

There is also another aspect I'm thinking about: we should remove the "experimental" prefix from the suite. And maybe add the "legacy" prefix to the current standard one?

[shaneutt]

Good call, I think:

we should remove the "experimental" prefix from the suite

Yes :+1:

and maybe add the "legacy" prefix to the current standard one?

The original intention was that the experimental one would simply merge in with the old one and become the old one, if that's feasible I think we should just go for that.

[mlavacca]

Good call, I think:

we should remove the "experimental" prefix from the suite

Yes 👍

and maybe add the "legacy" prefix to the current standard one?

The original intention was that the experimental one would simply merge in with the old one and become the old one, if that's feasible I think we should just go for that.

yes, I think it makes sense. I'll address the final migration in a different PR 👍