w3c
"Candidate Recommendation" is no longer an accurate term
The term Candidate Recommendation accurately and literally describes the state of a technical report that is expected to become a Recommendation.
Process 2020 explicitly contemplates technical reports remaining Candidate Recommendation at perpetuity, but being regularly revised (as a Candidate Recommendation Snapshot or Candidate Recommendation Draft).
The term Candidate Recommendation hardly describes these specifications, which will never become Recommendations.
Suggest either:
- selecting a different term to identify these technical reports that never become Recommendations; or
- change the Recommendation criteria such that these technical reports are considered Recommendations.
@palemieux I think it's hard to say that they will never become Recommendations. It's entirely possible that some specs that are under active evolution for many years in CR eventually stabilize enough to shift into REC status. RECs are still malleable, there's just a bit more process around making changes to ensure they continuously meet the additional criteria.
And there's still some value in viewing a REC, which requires two interoperable implementations, as the end state for a standard to strive for, rather than settling for a CR, which has lesser qualifications.
It's entirely possible that some specs that are under active evolution for many years in CR eventually stabilize enough to shift into REC status
What about specifications that a WG indicate will never become a REC?
[edit: for clarity]
I'm a bit ambivalent about this. Yes, for some groups, CR will be the intended end state, which makes the name not really appropriate. At the same time, for some other groups, CR will continue to fulfill the role it's always had, and for them it's still right.
So "Candidate Recommendation" isn't perfectly descriptive of all the case it'll be used in. But I am not sure that fixing this is actually a good idea. There's a bunch of existing expectations and institutional knowledge about what it means for a spec to be at CR, and changing names would mess with that.
It's not like there's no precedent with odd names for standards that are more grounded in history than in current practices. "Request For Comments" is somewhat weird and doesn't really reflect what those documents actually are. But we all know what it means, so it's there to stay.
The AB felt that we do not have the time to give this proper consideration for P2020, and it's a naming issue, not functionality. Keep open and discuss.
I think it is more than merely a naming issue. The CR SotD states:
Publication as a Candidate Recommendation does not imply endorsement by the W3C Membership. [...] It is inappropriate to cite this document as other than work in progress.
Does this still apply to technical reports that remain CRs at perpetuity?
In other words, should external organizations not reference specifications that are intended to remain CRs at perpetuity?
The comment about maturity is more important than the naming issue and I think it does matter. A document that doesn't get proper W3C review, but is published by a Working Group on their own with no broader endorsement is a Note. A document that does have that is a Recommendation. CR is neither one nor the other.
Yes, the status question is important. We are getting closer to the IETF 'RFC' status; formally 'requests for comment' but actually what the internet runs on. CRs have more status than Notes, just as standards-track RFCs do — they are on the way to becoming standards. But they are not Recs, just as most RFCs are not Standards.
In other words, should external organizations not reference specifications that are intended to remain CRs at perpetuity?
@palemieux I agree that this is more than a naming issue.
I don't think that anyone on the AB or Process CG wants to confer the same "status" to a CR as we do to REC. There are reviews that are not yet done (e.g. AC Review) so it is impossible to state that these documents have the consensus and endorsement of the W3C Community. There was an ac-forum discussion about how there is continued value to the REC status.
Clearly, there will be groups that are more focused on having patent commitments and an updated view of the WGs consensus. Indeed, some may not proceed to REC. If they take that choice, they are choosing that their document doesn't need to have endorsement of W3C. That will apply in some cases.
Your question about whether perpetual CRs may be referenced is therefore more complicated. Our documents are referenced all the time; even CG documents are referenced. So the issue is not whether they are allowed to be referenced. The question is what does a reference "mean".
Hence, if a group wants to reference a W3C specification that is a stable reference and has the consensus of the W3C Community, they should not reference a perpetual CR. But in other cases, groups might want to reference a perpetual CR.
@palemieux are you concerned that there may be specs that you want to reference as RECs, but will never get there?
It seems to me, that if some in the community need a spec to get to the full REC level, that requirement should be surfaced to the WG (e.g. in the AC review of the Charter); and in that case most WGs would respect that requirement and not leave things in CR in perpetuity.
@jeffjaffe My concern is with lack of clarity with what it means for a Technical Report to remain at CR at perpetuity. This concern applies to both (a) external parties that wish to reference the Technical Report and (b) W3 groups that need to choose the right path for their Technical Report.
In particular:
- does a perpetual CR represent the consensus of the W3C membership, or merely that of a WG Note, or something in between?
- is a perpetual CR a "work in progress" or a complete work for which new versions are likely to be published on a regular basis?
- does a perpetual CR need to demonstrate any implementation experience and/or have an associated test suite?
To be sure, I do not object to adapting the requirements for publishing consensus documents within W3C. I am particularly excited by the idea of simplifying the maintenance of Recommendations.
In discussion in the Process CG we realize that the SOTD needs work, see separate issues on that. We're not confident of our ability to edit the Process in 2020 but leave this issue open to collect thoughts on a future revision.
It seems to me, that if some in the community need a spec to get to the full REC level, that requirement should be surfaced to the WG (e.g. in the AC review of the Charter); and in that case most WGs would respect that requirement and not leave things in CR in perpetuity.
@jeffjaffe this is a worrying statement! I've never seen a Charter say "these are Rec track documents but we never intend to get to Rec". The working assumption of anyone reviewing a Charter has got to be that it is the WG's intention to get to Rec, rather than the other way around. Otherwise what's the point of having the document on the Rec track at all?
Analogously, if someone says "I plan to go to the store to buy ice cream" then would you respond to say "ok go ahead, but do make sure you return with ice cream"?
Concretely, it is very difficult to imagine any change to a Charter that would somehow add a stronger requirement for getting to Rec. Do you have something specific in mind?
I'm a bit ambivalent about this. Yes, for some groups, CR will be the intended end state, which makes the name not really appropriate. At the same time, for some other groups, CR will continue to fulfill the role it's always had, and for them it's still right.
Sorry to be blunt @frivoal but CR being the intended end state should not be acceptable. This is like saying "the intended state is permanent lack of clarity about the status of this work in the industry". W3C should never support this as an end state for a Rec track document. Even in the "Rec snapshot of continually updated document" model there are at least fixed points of clarity.
@nigelmegitt I think your response to Florian answers your question to me.
In your response to Florian you state that CR should never be the intended end state. But Pierre points out that some groups might do exactly that. In some respects, that happens today - we unfortunately have many specs that have languished in CR for too long.
My simple response is that if a stakeholder sees that happening - a spec stuck in perpetual CR - and if they need (e.g. for referenceability) that the spec go through full W3C endorsement - they have an opportunity to drive for such a commitment in Charter review. For example, they can object unless a particular mature snapshot be taken immediately to PR.
@jeffjaffe sorry I don't follow: how would Charter review, when there may only be an FPWD, or a mere concept for a specification, be an effective time to press for a push to Rec?
- Charter review is far too early to predict what will happen post-CR
- I doubt that there is any text that can sensibly be added to a Charter that will compel a WG to push towards Rec.
There's some relief here due to the slightly reduced typical Charter period, which means that re-Charters have to happen typically every 2 years or less.
Let's imagine that during a re-Charter, an existing spec that has been in CR for a long time were highlighted and a request made to add some language to the Charter asking the WG to push ahead for CR exit. That still doesn't make anything happen, as far I can tell.
@nigelmegitt I agree that for a new spec that is in FPWD, an AC rep should not complain that it has been in perpetual CR for too long. That is not the right time to object.
You are correct, that the time to catch it is when a spec has been in perpetual CR for too many years (from the pov of the AC reviewer). You are also correct that an AC reviewer cannot force a perpetual CR into REC in P2020 any more so than they can today. But I believe that if the AC community expresses their view that this is needed, that the W3C would find a way to bring the spec to REC - much as we are bringing WHATWG Review Drafts of HTML to REC.
Also, as @fantasai has pointed out, while the process to maintain a "living standard" at REC is heavier than it is for a CR, P2020 makes that possible too. Groups that favor a living standards approach have no less interest than everybody else in high quality specifications (and the criteria about the spec itself for getting to CR are as high as for REC, the difference is in terms of implementation experience and AC review). So once a spec is good enough to go to REC, and the rate of change is low enough because the technology is mature, and there is demand for a REC, it's not completely out of the range of possibility that "living standards" oriented group could still go to REC, and continue maintaining the spec in that state. It just isn't their priority (and isn't expected to be for quite a while). But that path remains open.
Given that (a) you can maintain in Rec as well as in CR and (b) anyone can ask/push for a Rec transition if it's needed, I would surmise that the reason a document stays in CR semi-permanently is because the WG and the community are comfortable with it.
There are some documents where there is a gentle trickle of updates and no obvious updates that are 'more significant' and would by themselves trigger a Rec. transition. But anyone can ask, and the AC could (effectively) insist.
@frivoal wrote:
once a spec is good enough to go to REC
@dwsinger wrote:
the WG and the community are comfortable with it
The problem I think we have is that specs are staying in CR, and being normatively referenced, when they clearly are not verified as being good enough to go to REC, because the CR Exit Criteria have not been met, but that the WG is comfortable with the situation. That does not equate to the community being comfortable as well, but it's hard to gauge the impact of this. My guess is that folk will vote with their feet and the W3C will lose relevance. That's why it's a Process issue.
I agree with @palemieux at https://github.com/w3c/w3process/issues/402#issuecomment-634408246 and I don't think the solution is to promote CR to being the new acceptable end state.
@frivoal wrote:
once a spec is good enough to go to REC
@dwsinger wrote:
the WG and the community are comfortable with it
The problem I think we have is that specs are staying in CR, and being normatively referenced, when they clearly are not verified as being good enough to go to REC, because the CR Exit Criteria have not been met, but that the WG is comfortable with the situation.
Maybe the document is good enough to go to Rec but the WG chooses not to. We shouldn't guess. They may feel the time isn't right, or that there is no particular time that is more right.
My guess is that folk will vote with their feet and the W3C will lose relevance. That's why it's a Process issue.
But there are other ways the community can vote with their feet as well, and the W3C lose relevance. The high nail today is that we suck at staying current, and doing maintenance, and the community is voting with their feet and finding ugly workarounds.
I don't think the solution is to promote CR to being the new acceptable end state.
I don't think we're either promoting it or deprecating it. We've had long-lived CRs for years; we're not changing that. We've had the ability to ask for a Rec. transition to occur; we're not changing that either.
Maybe the document is good enough to go to Rec but the WG chooses not to.
Has the process CG documented the reasons WGs do not to proceed to REC?
We've had long-lived CRs for years; we're not changing that.
If a perpetual CR in an acceptable end state, then the process should make this explicit.
Has the process CG documented the reasons WGs do not to proceed to REC?
Not formally, but in my experience WGs don't go to REC because of one or more of:
- There's known open issues that they intend to work on. We can't transition to REC with known unaddressed issues.
- Nobody is funding the requisite QA work to prove implementation spec-compliance and interoperability.
- There's known bugs / missing features in implementations that haven't yet been fixed, such that we don't have two interoperable ones [even if QA and issues are done already].
Taking a spec to REC is a lot of work. The AC can stamp its feet and say anything it wants in the charter, but a WG isn't going to take the spec to REC unless someone puts in the work.
However, if some community wants that work to happen, I'm sure any WG would be happy to support such efforts. Everyone wants better quality specs, tests, and implementations.
Maybe the document is good enough to go to Rec but the WG chooses not to. We shouldn't guess. They may feel the time isn't right, or that there is no particular time that is more right.
@dwsinger The suggestion here is that the spec has passed all the exit criteria but the WG deliberately delays requesting transition to PR. I can buy that for a matter of weeks, but not months or years. Are there any examples?
Taking a spec to REC is a lot of work. The AC can stamp its feet and say anything it wants in the charter, but a WG isn't going to take the spec to REC unless someone puts in the work.
@fantasai exactly right. My proposal at https://github.com/w3c/w3process/issues/406#issuecomment-636092886 would change the dynamic by moving the cost or penalty of inaction to the WG, where currently it is a reputational cost to W3C as a whole.
The hope would be that this encourages (funding of) work to address the open issues or the QA work, since the alternative is to reduce the value of the existing investment.
There's known bugs / missing features in implementations that haven't yet been fixed, such that we don't have two interoperable ones
If this is genuinely holding up specs then it sounds like the WG is holding their spec up to a higher standard than is required for getting to Rec - my understanding is that CR exit criteria must minimally demonstrate implementability of each feature, and that the implementation report need not consider interoperability per se. This could be another point to address via a clarification in the Process.
if some community wants that work to happen, I'm sure any WG would be happy to support such efforts. Everyone wants better quality specs, tests, and implementations.
Sorry, I think this is unrealistic. There's barely any chance that a community wanting a spec to be a Rec, e.g. to get a clearer view of its status, will step in and start doing QA work in it, since they will always expect the community that owns the spec to do that.
@dwsinger The suggestion here is that the spec has passed all the exit criteria but the WG deliberately delays requesting transition to PR. I can buy that for a matter of weeks, but not months or years. Are there any examples?
I think we've had cases where the spec. as it is, actually could go to Rec. but that the WG feels it shouldn't until some feature or missing functionality is addressed. I don't know. It would take team/chair conversations to find examples.
The SOTD doesn't ask that external organizations NOT reference; it says it's inappropriate to reference other than as a work in progress. That's true for an evolving CR, aka Living Standard.
There is no such thing as "perpetual"; all we know is the current state and the past; the future is uncertain.
I don't see a problem with a slightly quaint term; no more quaint than the IETF 'requesting comment'. And it's not even wrong; it could be proposed to move to recommendation.
I don't think either suggestion is appropriate: changing the name CR would be moving away from a term we are known for an know well. Calling CRs Recommendations would be... wrong/strange.
But we should pay close attention to the SOTD boilerplate. I believe that the SOTD is the place to fix this.
The SOTD doesn't ask that external organizations NOT reference; it says it's inappropriate to reference other than as a work in progress. That's true for an evolving CR, aka Living Standard.
@dwsinger These two sentences seem to highlight a possible difference of world view: getting to alignment might really help us to move forward:
-
I don't know any other standards organisations that include normative references to unverified works in progress as a matter of course (I do know of SDOs that don't have any formal requirements but leave it to contributors though, like ETSI). Therefore, from a practical perspective, saying "inappropriate to reference other than as a work in progress" is effectively asking not to reference.
-
"evolving CR" != Living Standard: "living standards" hold each normative change up to a higher standard before accepting it than CR does. Specifically, tests are needed as well as either implementations that pass them, or commitment to implement. Whereas no testing at all is required to enter CR. This is a significant difference.
1. I don't know any other standards organisations that include normative references to unverified works in progress as a matter of course
We already support normative references to WHATWG HTML Living Standards when that is the most appropriate reference.
We already support normative references to
@jeffjaffe I'm talking about other organisations referencing W3C standards. But if you're trying to highlight existing weaknesses in W3C, then yes, that is a true statement.
Ah, I missed a resolution from the Call on 27 May 2020
RESOLUTION: we keep 402 without Process changes this year, but turn our attention to improving the SOTD