kubernetes-sigs
WIP: Remove stale implementations from implementation list
What type of PR is this? /kind documentation
What this PR does / why we need it:
The goal of this PR is to help users understand which viable options they have when choosing an implementation.
Recently, I put on my 'user hat' and was very frustrated to find that many implementations I had chosen were unusable. Some had only support for ancient versions like 0.4, while others point to an entirely empty repository. We should give users an up to date list of implementations to help guide them.
My goal here is not to force out any implementation -- I think a large active list/community is ideal. It is only to weed out implementations that are entirely dead.
That being said, I think this PR goes too far. My initial criteria I came up with "Has ever submitted a conformance report" I assumed would be easily met, yet it excludes 17/30 implementations!
So I think before merging this we will want to either: a) Come up with a new criteria. b) Give implementations a N month warning to send a conformance report or be removed
To re-iterate, the criteria I am proposing is not "passes conformance tests", merely has submitted one, which is a very low bar. I also included "recently", which I interpreted as "since 1.0 (when we first started collecting reports)" for now. In the future, we would likely make this something like in the past year/past N releases/etc.
Note: if we are to move forward with this, we also need to remove the stale descriptions of the removed projects. I will do that once we come to some consensus.
Which issue(s) this PR fixes:
Fixes #
Does this PR introduce a user-facing change?:
Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.
[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: howardjohn Once this PR has been reviewed and has the lgtm label, please assign robscott for approval. For more information see the Code Review Process.
The full list of commands accepted by this bot can be found here.
Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment
Appreciate the nuance in rationale and agree that at this point ensuring we're guiding users to active implementations to set them up for success will ultimately be beneficial to the Gateway API project as a whole over just demonstrating widespread vendor adoption to show initial traction.
As a first step we've discussed (cc @robscott) the idea of "elevating" or otherwise emphasizing conformant implementations before (maybe as you suggested starting with just impls that have submitted reports at all is a fine place to begin). We should probably also do some proactive outreach here, identifying which users initially added each project to the list and tagging them for visibility (and to hopefully confirm abandonment or encourage submitting reports) before removal.
What if we aim for the following:
- Implementations that have submitted a conformance report for one of the last N releases are highlighted at the top, maybe with some kind of visual indication for the versions they've submitted conformance reports for.
- Implementations that don't qualify for 1 fall to the bottom of the page in a new section, title TBD, with a note that implementations in this section will be removed on some future date (maybe August 1) if they fail to submit a conformance report prior to that date.
- All new implementations of the API must also submit a conformance report.
I'm not really sure on 3 - I do think it's useful to be able to highlight the implementations that are making progress towards conformance, but I do want to avoid implementations that get stuck in a WIP state.
I've seen implementations advertise Gateway API support without running a single conformance test. When I ran conformance tests myself against said implementation it was painfully clear it wasn't conformant for even core features.
This breaks the guarantee of compatibility and portability for users. Hence why I don't think we should even advertise implementations that are 'working towards conformance' because it's meaningless and not useful information for users.
In practice implementations shouldn't use the GatewayAPI trademark without submitting a report.
I'm not really sure on 3 - I do think it's useful to be able to highlight the implementations that are making progress towards conformance, but I do want to avoid implementations that get stuck in a WIP state.
IMO there is no excuse to not have a report. I am fine with implementations being listed that fail every test, as long as they have a report - and the page will indicate as such
It would be good to require at least v1 conformance by end of year. Do we want to rule out implementations that never had a use for some of the newer standard features that are not table-stakes features? For example, GRPCRoute or percentage-based mirroring?
It would be good to require at least v1 conformance by end of year. Do we want to rule out implementations that never had a use for some of the newer standard features that are not table-stakes features? For example, GRPCRoute or percentage-based mirroring?
My proposal is that you just need a conformance report. You can fail every test
echoing feedback from community meeting today:
How about we do something like:
- Decide that every new implementation that gets in, submit an associated conformance report in the same PR?
- Set a deadline for existing implementations, if no conformance report is submitted by then - they are going to be removed
- Provide guidance/expectations about the versions and conformance, for example, we require conformance report for version latest-3 of the API at least at any given point.
Oh, and ideally we provide an automation that will do the removal and not relying on manual reminders
PR needs rebase.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.
I think this is made obsolete by the changes in #3863, so I'll close. @howardjohn please comment here if you disagree.
