Discussion Thread: Priority of Constituencies

[sarahghp]

In discussing updates to documentation, we discussed adding an explicit priority of constituencies. We've identified some constituencies, but not their order. Here is a place to discuss that.

Suggested constituencies include:

• Implementors

  • Big browsers
  • Small browsers
  • Parsers/transpilers
  • Polyfills
  • Server-side

• Test writers

  • Casual test writers
  • Test writers already involved with one of the other constituencies (implementors/proposal champions)

• Proposal champions

• Maintainers group

Specifically excluded:

• Implementations that are just lexing, e.g. code highlighters.

Unsure

• People who want to use Test262 as a repository of code examples to learn about how the language works (or what the specification means) ▶️ Related discussion

[sarahghp]

Strawlady ordering:

1. Test writers
2. Implementors
3. Proposal champions
4. Maintainers group

[jugglinmike]

Here's my take:

1. Implementers
2. Proposal champions
3. Test writers
4. Maintainers

Here's why:

My feeling is that the ultimate goal of Test262 is implementation correctness. People might use the tests for all sorts of reasons, but I can't think of any more important than verifying claims to compliance. That's why, to my mind, implementers are the highest-priority constituents.

Since "compliance" only has meaning in the context of a specification, proposal champions seem like the next most important constituents. (We might even expand that category to include authors of spec patches--maybe "specification contributors.")

It seems right to me to recognize that the maintainers' perspective is relevant, and that as stewards, they should give preferential treatment to everyone else. Test writers participate in a similarly-altruistic capacity, so some folks might not even see the value in distinguishing between people writing tests and people maintaining Test262. I think they deserve recognition because they don't enjoy the same decision-making power as maintainers, but I don't think this puts their needs above the people defining and implementing the language.

[sarahghp]

The reason I think test writers should be the first in the list is that Test262 does not grow without them and would therefore become useless to implementors — no tests, no correctness check. The writers are what makes the project go.

I also think that as the people with the least power in the project, but arguably one of the most important roles, test writers are analogous to the users in the case of browsers, and by putting their needs first, we focus the project on being welcoming and sustainable.

[ptomato]

In most of my projects I try to make a distinction between new / new-ish contributors and recurring contributors, and prioritize the former above the latter and maintainers. That's because I believe that the reception that a new or casual contributor gets, is the biggest determinant of whether they stay or move on. So, for example, if I get a pull request from a new contributor that is correct but not organized or formatted the way I want it, I'll likely just merge it and fix up the organization later.

As for maintainers, it sounds altruistic that we should put ourselves last, but in practice I do a lot of things in other projects where I'm a maintainer, that prioritize my time above recurring contributors who aren't maintainers, for the sake of sustainability. For example, while I'll happily fix up linter errors from someone's first few contributions and merge them speedily, once someone's "hooked" I'll generally feel free to say "hey, would you mind fixing the linter errors" and mostly ignore the PR until they do. The flip side to this is to be really welcoming to recurring contributors who are interested in the increased responsibility of a maintainer.

So my take would be:

1. Potential, first-time, or casual test writers
2. Maintainers
3. Test writers

I'm not sure where to put proposal authors and implementations in this list. (Proposal authors and implementors who are not writing tests, that is. They are partially already covered by "test writers".)

I'd say we should, above all, never make a decision that would lead to implementations being unable to rely on the correctness of test262. So that would place implementations at the top of the list. On the other hand, I could see a hypothetical situation where we might want to make some change for the sake of maintainability that requires implementations to update their runners, which of course they wouldn't want to do. While I don't think such a decision should be taken lightly, I don't think implementations ought to take priority in that case.

Proposal authors who are not also test writers or potential test writers: what should be our involvement with that group? We should make it easy for proposal authors to become test writers (see no. 1 in my list.) But if there is a hypothetical Stage 3 proposal for which the champions have disavowed any interest in writing tests no matter how inviting we make it, what is our obligation to them?

[ljharb]

Implementation correctness definitely seems to me to be number 1, but I'd rank implementor convenience much, much lower.

[sarahghp]

I wonder if it makes sense to think less about the constituencies as groups of people but break it up more into concepts (since I see us doing it anyways 😆). So it might be something like:

1. Implementation correctness / coverage
2. New test writers' experience
3. Implementor experience (this could be broken up into runners & ability to understand status, say)
4. Maintainer experience
5. Seasoned test writers
6. Proposal champions

[ljharb]

I like that.