aria-at
aria-at copied to clipboard
w3c
Developing a welcoming plan for new community group members
If people are to use the test runner to execute tests and review them in-place, there are certain steps they need to carry out: creating a GitHub account, joining a specific GitHub org/group to get access to the test runner, etc. These steps currently aren't documented/defined and we need a "Getting Started" page to help us all out. We should use this thread to first actually define and enumerate the needed steps.
As discussed on the July 15 CG call, the welcoming experience for new community group members should include:
- Documentation
- Support
- Mentorship
- Onboarding
- Signup & Permissions
- Inclusive Meetings
- Introductions
Relevant meeting minutes with ideas: https://www.w3.org/2021/07/15-aria-at-minutes.html
As a brand-new member of ARIA-AT, I would really appreciate a new member orientation. I have attended a few meetings and still have no idea what I am supposed to be doing or how I am able to contribute. Some guidance and instructions would bring me up to date on the work the group is doing and allow me to feel useful and contribute somehow.
I am copying here an ARIA-AT overview that I sent to folks who expressed interest on last week's CG call. I hope to work with the group to turn this description into better documentation.
The root of the issues in the last few weeks is that we have an old version of our process that is mostly outdated, and the new version isn't quite ready yet. I'll explain in a bit more detail, but first I'll share some context about what type of work different people are doing.
ARIA-AT's goal is to improve screen reader interoperability by testing various user interface patterns across different screen readers and browsers. Currently, the user interface patterns come from the ARIA Practices Guide or APG examples. We write test assertions that codify the expected behavior for a given APG example. This is the basis for all of the work of the community group.
For example, APG publishes an example called Combobox (Select Only) that implements a single-line textbox and an associated pop-up element for helping users select the value of the textbox. We write a "test plan" composed of 10-30 individual "tests", that each cover a granular element of user interaction. For the "Combobox (Select Only)" test plan, one test is "Navigate backwards to a collapsed select-only combobox in reading mode." This test in turn contains a handful of "assertions" that specify what the screen reader should convey after a given user keyboard command. For the above test, one assertion is "Role 'combobox' is conveyed" after the keyboard command "Shift+F". When we talk about writing tests in the community group, you might hear about each of these levels of hierarchy.
Just to give a sense of scope, we have published 16 test plans for APG examples, each with 10-30 individual tests. About 10 of those test plans were published in the last 2 weeks. Hence, the community group's excitement about onboarding new volunteers and reviewing these new test plans. All of the content for the test plans can be viewed and reviewed at this raw HTML link. The interface is spartan and read-only, but the content is all there.
Meanwhile, there is another workstream that is building a webapp to record and display test results. This webapp is called ARIA-AT App. The App has a user-friendly interface to assign test plans to testers, run test plans, collect the results of those runs, review results, and publish results in tabular format as reports.
To return to the initial issue, the existing version of ARIA-AT App is now outdated, but the new version is still a few weeks away from being published. This obviously complicates the process of onboarding. I think you've heard our hesitation to spend a lot of time and effort documenting the soon-to-be-replaced version of the app. On the other hand, it's frustrating to not be quite ready to onboard anyone to the new app yet.
I would suggest that we spend the next 2-3 weeks (before the new app is deployed) coming up with a plan for where documentation / onboarding should live, how to join the W3C Community Group (more on that below), how to welcome newcomers, how to improve communication and meetings for newcomers, and how to explain the jargon around tests, test plans, assertions, and APG examples.
As soon as the new app is deployed next month, we can then extend the documentation / onboarding experience to cover the webapp and the workflows that it unlocks.
Hi there,
I will no longer be a member of this group. Please take me off of the mailing list. Thank you.
Crystal Scott
On Thu, Jul 29, 2021 at 8:42 AM Seth Thompson @.***> wrote:
I am copying here an ARIA-AT overview that I sent to folks who expressed interest on last week's CG call. I hope to work with the group to turn this description into better documentation.
The root of the issues in the last few weeks is that we have an old version of our process that is mostly outdated, and the new version isn't quite ready yet. I'll explain in a bit more detail, but first I'll share some context about what type of work different people are doing.
ARIA-AT's goal is to improve screen reader interoperability by testing various user interface patterns across different screen readers and browsers. Currently, the user interface patterns come from the ARIA Practices Guide or APG examples. We write test assertions that codify the expected behavior for a given APG example. This is the basis for all of the work of the community group.
For example, APG publishes an example called Combobox (Select Only) https://w3c.github.io/aria-practices/examples/combobox/combobox-select-only.html that implements a single-line textbox and an associated pop-up element for helping users select the value of the textbox. We write a "test plan" composed of 10-30 individual "tests", that each cover a granular element of user interaction. For the "Combobox (Select Only)" test plan https://w3c.github.io/aria-at/build/tests/combobox-select-only/index.html, one test is "Navigate backwards to a collapsed select-only combobox in reading mode." https://w3c.github.io/aria-at/build/tests/combobox-select-only/test-02-navigate-backwards-to-collapsed-select-only-combobox-reading.html?at=jaws This test in turn contains a handful of "assertions" that specify what the screen reader should convey after a given user keyboard command. For the above test, one assertion is "Role 'combobox' is conveyed" after the keyboard command "Shift+F". When we talk about writing tests in the community group, you might hear about each of these levels of hierarchy.
Just to give a sense of scope, we have published 16 test plans for APG examples https://w3c.github.io/aria-at/build/, each with 10-30 individual tests. About 10 of those test plans were published in the last 2 weeks. Hence, the community group's excitement about onboarding new volunteers and reviewing these new test plans. All of the content for the test plans can be viewed and reviewed at this raw HTML link https://w3c.github.io/aria-at/build/. The interface is spartan and read-only, but the content is all there.
Meanwhile, there is another workstream that is building a webapp to record and display test results. This webapp is called ARIA-AT App https://aria-at.w3.org/. The App has a user-friendly interface to assign test plans to testers, run test plans, collect the results of those runs, review results, and publish results in tabular format as reports.
To return to the initial issue, the existing version of ARIA-AT App is now outdated, but the new version is still a few weeks away from being published. This obviously complicates the process of onboarding. I think you've heard our hesitation to spend a lot of time and effort documenting the soon-to-be-replaced version of the app. On the other hand, it's frustrating to not be quite ready to onboard anyone to the new app yet.
I would suggest that we spend the next 2-3 weeks (before the new app is deployed) coming up with a plan for where documentation / onboarding should live, how to join the W3C Community Group (more on that below), how to welcome newcomers, how to improve communication and meetings for newcomers, and how to explain the jargon around tests, test plans, assertions, and APG examples.
As soon as the new app is deployed next month, we can then extend the documentation / onboarding experience to cover the webapp and the workflows that it unlocks.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/w3c/aria-at/issues/420#issuecomment-889253859, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUMON5QSAXWGFKY2SRZQILDT2FZFDANCNFSM42RV637Q .
-- Crystal L. Scott 503-878-0208
Update wiki page on running a test plan: https://github.com/w3c/aria-at/wiki/Running-a-Test-Plan
We discussed this issue during today's Community Group meeting and touched on additional ideas for improvements. The minutes are available here.
