act-rules.github.io icon indicating copy to clipboard operation
act-rules.github.io copied to clipboard

Boundary / Verbose examples

Open Jym77 opened this issue 1 year ago • 3 comments

Playing around the idea of boundary examples as discussed in #2164.

This adds one "boundary example" to some of the rules, showing how the rules are partial check and stuff that is inapplicable or even passing might still fail the SC (or other requirements).

These "boundary examples" must not be part of the normal examples that are required for tools to claim "consistent implementation" of the rule. Thus, tools can be more clever than the rule (i.e. fail these cases), but still have a consistent implementation.

This PR only shows a possible way to show these examples: in a collapsible section after the normal examples (so they are still together with the rule itself, but do not take space for people not interested in them). If we decide to go that way, we would need to build the website code (and tweak the validation) to automate part of this process.

I do believe that this won't require a format update since the format doesn't prohibit "extra" examples. Another possibility would be to place the boundary examples in the Background, but that makes them show a bit earlier in the text of the rule (even if the section is collapsed).

Closes issue(s):

  • N/A

Need for Call for Review: This will require a 2 weeks Call for Review (new concept)


Pull Request Etiquette

When creating PR:

  • [X] Make sure you're requesting to pull a branch (right side) to the develop branch (left side).
  • [X] Make sure you do not remove the "How to Review and Approve" section in your pull request description

After creating PR:

  • [X] Add yourself (and co-authors) as "Assignees" for PR.
  • [X] Add label to indicate if it's a Rule, Definition or Chore.
  • [X] Link the PR to any issue it solves. This will be done automatically by referencing the issue at the top of this comment in the indicated place.
  • [X] Optionally request feedback from anyone in particular by assigning them as "Reviewers".

When merging a PR:

  • [ ] Close any issue that the PR resolves. This will happen automatically upon merging if the PR was correctly linked to the issue, e.g. by referencing the issue at the top of this comment.

How to Review And Approve

  • Go to the “Files changed” tab
  • Here you will have the option to leave comments on different lines.
  • Once the review is completed, find the “Review changes” button in the top right, select “Approve” (if you are really confident in the rule) or "Request changes" and click “Submit review”.
  • Make sure to also review the proposed Call for Review period. In case of disagreement, the longer period wins.

Jym77 avatar Apr 25 '24 12:04 Jym77

I believe there is no need to clarify the outcome of each specific example for the specific rule (passed, failed, inapplicable) within a heading.

By definition, boundary examples "are not normative and are not required for consistent implementations." Therefore, in my opinion, and as recommended by @dan-tripp-siteimprove, even though I understand the intention to emphasize whether the current scenario is passing or inapplicable, we should avoid distinguishing this in the headings hierarchy to prevent confusion.

The reason why an example is an edge case, in my opinion, should be provided in the description of the example, leaving just "Boundary Example 1," "Boundary Example 2," etc., as titles.

giacomo-petri avatar Jun 19 '24 08:06 giacomo-petri

And sorry, I forgot to mention the most important thing: I really love this idea! We should definitely move forward with this idea!

giacomo-petri avatar Jun 20 '24 06:06 giacomo-petri

Just leaving my 2 cents.

  1. I don't think we should put details / summary in here. I'd rather have that be added by our build tool. It's less error prone that way, and means we can more easily make changes to it in the future.
  2. I think we should call these "Miscellaneous examples". As I understand the conversation, we'll want to use this for a variety of different kinds of examples. The only thing that they have in common is that they're not passed / failed / inapplicable examples.
  3. I feel we should use different H3s / H4s here. Calling these a passed example, or an inapplicable example doesn't set it apart from the rest of the rule sufficiently in my opinion. People will get confused by what these passed examples mean compared to the other passed examples. We should call them something else. There are lots of things we can use here, like "Out of scope example 1", "Accessibility support example 2", "Explicit semantic role example 3", "Shadow DOM example 4". etc.
  1. Definitely. This was just to show how it can look like.
  2. 👍
  3. I think it can be good for tools vendors to have a way to retrieve these cases and use them for validation purpose. This will require us to provide the rule's result in a way that the build tool can understand and forward. But I agree that the names can be confusing.

Jym77 avatar Jul 02 '24 08:07 Jym77