publishingcg
publishingcg copied to clipboard
w3c
Is 0 tolerance for accessibility conformance practical?
As per the design of WCAG 2.x and EPUB Accessibility 1.x, The producer need to ensure that all Content Documents meet all the accessibility requirements for claiming conformance to EPUB Accessibility 1.x or WCAG 2.x. The publishers find it challenging to meet this requirement of nearly zero tolerance, especially when they are publishing big publications. I am not sure if we can do much about this at this stage, this issue will be addressed by WCAG 3.0. But it is worth starting a discussion. Please see the minutes of accessibility task force held on August 12, 2021 for the initial round of discussions: https://www.w3.org/publishing/groups/epub-wg/Meetings/Minutes/2021-08-12-epub-a11y#section3
I'll try to express some of my thoughts:
- EPUB Accessibility specifications allow to align to two levels of WCAG 2: level A and level AA
- publishers can align to level A, if level AA is not achievable
- if tolerance 0 is exaggerated, what is the appropriate level of tolerance to declare that a product is accessible?
- at the moment (with WCAG 2) I think the only way is to align to the different levels defined by the specifications (A, AA, AAA), with WCAG 3 - instead - it will be possible to better express the degree of accessibility.
- if publishers are afraid of being "accused" of an accessibility error:
- if it is a local error (wrong alt-text, text not correctly tagged, ARIA role improperly used) it can be treated in the same way as typos: the user reports the problem to the publisher (or to the certifying body) which commits to remedy it
- if the error is structural (e.g.: all fourth-level headers are tagged as
<p>, or all mathematical formulas are images without alt-text), here we come back to the previous question: what is the minimum level to declare a publication accessible?
Finally, I would like to add that, as Fondazione LIA, we validate all digital publications of our member publishers and - in case of accessibility problems - we report the problem to the publisher, asking them to fix it. Until accessibility issues are resolved, we do not certify the title. This for us (and for the publisher) is a guarantee of quality.
Where WCAG 3 ends up is still a bit of a mystery at this stage, but it is centred around trying to handle this issue of perfection or nothing.
Currently, it would do away with Level A conformance, as Bronze is equal to Level AA. But what it does, or is trying to figure out how to do, is weigh the impact of non-conformance against user needs. Some requirements will remain absolute -- any failures will fail the content -- but less critical requirements may pass even if there are a few errors. Publications would get scored for each method so users could also review these numbers to see where the flaws are.
But while I can sympathize with the inflexibility of dealing with WCAG 2 conformance, we really shouldn't move ourselves out ahead of the Silver group, especially when it comes to conforming with WCAG 2. If we don't follow the same conformance rules, then we can't claim that anyone is meeting WCAG.
This sounds a lot like the discussion we had when we started the 1.0 revision and were considering a new level of our own for content that met some basic requirements of WCAG but not everything. Given the sometimes legal ramifications of not adhering to WCAG, it probably remains best not muddy the waters of what publishers are achieving.
The issue was discussed in a meeting on 2021-08-26
- no resolutions were taken
View the transcript
1. Continue discussion on is 0 tolerance for accessibility conformance practical?
See github issue #1767.
Avneesh Singh: we had a long discussion about this issue during last call
… Charles suggested to use a string "significantly" conforms to
… but then we had some comments on the issue
John Foliot: I like the idea to use "significantly conforms to"
… I the VPAT you can describe it, to tell that perfect is a goal
… but achieve it is complicated
… in the USA an accessibility problem can goes to a legal issue
Matt Garrish: I think it's ok for a publisher to tell it in the accessibility summary
… I have concerns about writing in specs what it means to be significantly accessible
Avneesh Singh: do you mean that you can put whatever you want in the conformsTo field?
Matt Garrish: yes
John Foliot: I'm active in the Silver WG, we are working on the score system
… in conformsTo may we link so a self-published accessibility statement (for publishers)?
Matt Garrish: we have certifierReport metadata for doing this
… it's optional
John Foliot: the conformsTo metadata instead is not optional, it is mandatory
… what about about making only one in conformsTo and certifierReport mandatory?
Matt Garrish: actually conformsTo is not mandatory
Bill Kasdorf: to end users what is most interesting is what it doesn't conform
… it would be useful to express it
Gregorio Pellegrino: for local legislation the conformance is important
… also in metadata we don't have ways for telling which accessibility issues are present in a document
George Kerscher: I don't think we should get out from Silver
John Foliot: +1 to George
John Foliot: WCAG 3 is "far off" IMHO
Avneesh Singh: is it a problem to have conformsTo linking to WCAG and then in the accessibilitySummary telling that there may be issues?
George Kerscher: I think we should not worry about legal issues
John Foliot: my background tells me that think about legal risks is important
… I think that standards should not enable contractions
… I think we should balance it
George Kerscher: what does the the UX metadata guidelines tell to display when conformsTo is empty?
Avneesh Singh: nothing or unknown?
George Kerscher: so you can use certifierReport instead conformsTo
John Foliot: some certifier require to have conformsTo metadata for their certification
… so it is not "optional" for being certified
… I think we need a formal reference to link to
Matt Garrish: we can add in the specs that if partial conformance is present, then the certifierReport is mandatory
… in the conformanceTo we may use EPUB11-WCAG21-A-PARTIAL
John Foliot: +1 Matt - that's what I personally would like to see: addressing a real-world issue with a "standardized" solution
Bill Kasdorf: +1 to Matt
John Foliot: I would also suggest to define a "standard" way to make certifier reports
… VPAT may be a solution
Matt Garrish: sure, we can recommend it
Gregorio Pellegrino: for me the certifierReport should be per publication and not for all publications
Matt Garrish: I don't know
Bill Kasdorf: I'm favor for this approaches, but I see many VPATs that are false
… a lot of publishers are not able to read VPATs
John Foliot: to be clear, I am not advocating using a VPAT per-se, but rather look at VPATs as a model for conformance reporting
Bill Kasdorf: I think that have a report per publication is useful plus a report from the publisher telling on what they're working
Wendy Reid: I'm concerned about telling that you must have to create a report for titles
… I think trade publishers would not use it, because it is not meaningful for retailers
Avneesh Singh: why not relaxing the report, if we have accessibilitySummary
… ?
Matt Garrish: yes, I think there are different ways to achieve it
Avneesh Singh: I think we had an excellent discussion, Matt can you now propose changes for the specs?
John Foliot: I think there are differences between prose versus machine-readable metadata
… so we should make the distinction clear in the specs
… for example telling the conformsTo MUST be an URL
Matt Garrish: we had this discussion for EPUB Accessibility 1.1
… we had issues for complex URLs
… I think that we can find a way using conformsTo plus certifierReport
The issue was discussed in a meeting on 2021-09-09
- no resolutions were taken
View the transcript
1. Is 0 tolerance for accessibility conformance practical? (pr epub-specs#1767)
See github pull request #1767.
Avneesh Singh: mgarrish has done a lot of work in PRs
… we had a lot of discussion on WCAG zero-tolerance. We've decided on an option to address this in an editor's meeting last week
Matt Garrish: this is to address almost-conformance. We're not trying to create something new. Rather have publishers be able to provide a disclaimer that says they did their best, but there could still be minor imperfections
… this PR adds disclaimer property. Nothing similar already exists in Schema.org
… this will not be a requirement for publisher to use this. Only an option
Avneesh Singh: backwards compatibility was important to us in picking this solution. And minimizing disruption to existing workflows
Will Awad: q
Will Awad: is this disclaimer different from the a11y statement that a publisher would put on their website?
Avneesh Singh: this would be in the epub metadata
Matt Garrish: the reality is that things slip through when a11y conformance is a combination of machine and human checking
… so publisher is saying that to the best of their ability the epub meets the standard they are claiming
Will Awad: so will it be a failure of conformance if they leave out disclaimer?
Charles LaPierre: so you have a conformance statement. But what we're doing in addition to a11y summary statement is to have a more formal disclaimer property that publisher can add
… for Benetech certification conformance is the same, but this will be one more thing that certified publishers can put in their epubs (to mitigate liability, etc.)
John Foliot: so the statement says that the epub is WCAG compliant, but then the disclaimer says "not really"?
George Kerscher: when I contacted Bruce Bailey from 508 access board I asked about this
… and in 508 the standard is black and white as well
… he said if you look hard enough, you will be able to find some defect
… but the industry has come to peace with this issue and don't have any problems with that
… our disclaimer says exactly that, and no more
Charles LaPierre: and we also have to be backwards compatibility with existing epubs, which this solution is
John Foliot: but the statement and the disclaimer are contradictory right? So at what point are there too many non-compliances?
Matt Garrish: not sure that there is a contradiction. But disclaimer acknowledges that we live in an imperfect world.
… and it creates an avenue for reporting when incidents of non-compliance are found
John Foliot: I understand, but, I still think that the disclaimer is contradictory. WCAG is explicit in that it requires 100% perfection.
… WCAG3 will acknowledge this issue of 100% compliance or nothing
… but right now, I'm concerned that in the real world a lot of organizations won't put in the work to do a11y compliance anymore
Tzviya Siegman: my concern is that a disclaimer like this doesn't accomplish anything. If the point is to make sure that publishers are covered legally, I'm not sure this does it
… i'd have to check with out legal team to see if something like this is useful
… and if not, then I don't see what the purpose of this is
John Foliot: +1 Tzviya
Charles LaPierre: the disclaimer adds credibility for publishers that choose to go out and get 3rd party certification
Matt Garrish: if you don't disclaim it, you still have a false claim
… if we don't have anything, are we any better off?
John Foliot: part of the answer is how we apply the metadata. One way is to ref to a URL. And the other is using a meta-element that is a text field.
… 2nd way i think is a path towards a more accurate statement
… from a legal perspective VPATs have a ability to say partially supports
Tzviya Siegman: part of my concern is that the document doesn't say how to use this
… maybe if you want to say "epub is WCAG compliant except for this particular criteria in this particular circumstance"
… seems like there is a lot of wiggle room here
George Kerscher: if JF and tzviya think this disclaimer does nothing, maybe we should get rid of it and leave things they way they have been. There are a lot of documents out there that are reporting WCAG AA, companies that sell to universities that claim WCAG AA, and if you look close enough you'll find that many of those are incorrect statements
… the industry has come to terms with this, and maybe its not up to us to solve it
Wendy Reid: second tzviya that we should be clear about what is allowed
… but i think this has value particularly in the retail space. When retailers start to display a11y metadata, the disclaimer is going to be important to both retailers and users. Users should be made aware of that before it gets escalated to a customer care call.
… retailer gets accused to having a problem when the issue is in the epub
… this is already happening, and we don't even display a11y metadata yet
… this gives users as much information as possible
… not just a legal issue, but empowers users to know what they are purchasing
Avneesh Singh: one option for doing this is to include such information in the a11y summary
Gregorio Pellegrino: we have to define what it means to strive for WCAG AA if, for example, there is something in your document that don't meet WCAG A
Matt Garrish: i pictured this mostly being used for errors that slip though, where nobody could have caught it
… once we get to partial conformance you're not complying to epub a11y spec either
… so the other option is that we accept that epub just shouldn't have a dc:conforms statement
… include details in VPAT
… for content that has known and potentially serious issues with it, then let's not try to solve that from a conformance perspective
John Foliot:
<meta ... content="my VPAT.html">
John Foliot: maybe part of the problem is that we don't want to be pointing to a specific WCAG
… if we use a meta-declaration (see above), could we open the spec to point to, for example, a VPAT statement?
… re. george's statement of institutions that already claim conformance, some EDUs want very richly annotated VPATs
… so in some ways a short, patterned conformance statement is less useful than a detailed document
Avneesh Singh: there are some mechanisms already built into metadata to deal with this
Matt Garrish: doesn't seem right to say that VPAT is a conformance statement. A VPAT is a report of how you conform to something else
… there's nothing something can determine from that sort of URL
… you can put it in if you want. The spec can't prevent you from doing that. But not sure we want to put that in the spec.
Avneesh Singh: certifierReport is the right place for that sort of thing
… to summarize, we don't like the disclaimer. It may be useful, but not for the purpose for which it was created.
… let's leave it for now
… mgarrish are you satisfied with the certifierReport and other metadata that can meet this need?
Matt Garrish: maybe we can make it clear that even if you aren't providing a conformance claim you can still provide a link to a certifierReport
Avneesh Singh: i think it would be nice to explain that these can be used in this way
George Kerscher: if we have no conformance statement you could still include certifiedby and you could clarify in your a11y summary what you conform to
… when the metadata is shown according to the UX guide, for conformsto it would say none provided, right?
Avneesh Singh: yes, for now, but we still have to revisit the UX guide
John Foliot: the only other potential problem i see is WCAG3
… from the start WCAG3 is being built for various levels of conformance: bronze, silver, gold
… and a scoring mechanism still being sorted out
… so we're going to have to be able to declare which which level an epub is at
Avneesh Singh: no, we've decided that we're not going with this disclaimer route, so we don't need to state closeness of conformance
… and if you don't completely conform, then the epub shouldn't include a conformance statement
Charles LaPierre: I'm not in favor of pulling the disclaimer. The disclaimer puts publishers at ease, acknowledges forgiveness for human error, provides route for reporting. I think having it is better than not.
Avneesh Singh: let's continue discussion in issue tracker, as I don't think we can resolve on this issue today
Charles LaPierre: I think we should vote on this today
Avneesh Singh: per W3C practice we need consensus
… at 2 months away from CR, and with some publishers missing, I'm not sure a vote is a good idea right now
Charles LaPierre: +1 to Matts suggestion.
Matt Garrish: we don't need to make this either/or. Disclaimer is optional. But we can still add explanatory statement that people can leave out conformsto and link to VPAT.
John Foliot: I like that route. Choice is good.
Matt Garrish: from an editing perspective I can let you look at both and see how you feel
Avneesh Singh: so, for people who were opposed to this, does the non-compulsory nature of the disclaimer change your mind?
Tzviya Siegman: I think it needs to be clearer how the disclaimer is intended to be used
Matt Garrish: sure, I can try to work on that
George Kerscher: in my mind bottom line is that there is no catastrophic failure
John Foliot: even though disclaimer is optional, I think we're going to see it all the time. We're back to the contradictory statements.
Avneesh Singh: so it would serve the purpose of allowing the publisher to make a more accurate statement, rather than providing any legal certainty
Matt Garrish: and we can clarify that putting in a disclaimer doesn't absolve you from legal liability
… do we want to rename it to contact information rather than disclaimer?
… I think ONIX does a similar thing
John Foliot: and that contact info is baked into VPATs as well
Gregorio Pellegrino: if its only simple text we could put it in the last sentence of the a11y summary
Charles LaPierre: the whole reason for this was because publishers were worried about saying conformsto WCAG AA. I don't think adding a contact us address that issue.
John Foliot: +1 to Charles, we need both
Avneesh Singh: it seems we don't have consensus. So let's continue discussion in issue tracker.
… and let's remember that we are just 2 months away from CR, so we can't make too many changes this time around
I've been trying to reason out another approach to this since we met yesterday, and am now wondering if we've overlooked the most basic means of expressing conformance to the specification -- through discovery metadata alone.
What if we formalize discovery metadata-only publications as a conformance measure, so add an additional string like:
EPUB-A11Y-11_DISCOVERABLE
This identifier would avoid having to define a negative sounding name like "partial" or "substantial" or "incomplete" or "failed". It would also add a conformsTo statement to cover:
- when only discoverability metadata is included, which is the absolute minimal requirement of the specification
- when a publication fails success criteria and a claim of WCAG conformance can't be made
- when a publication passes but the publisher does not want to make an explicit claim of conformance to WCAG (i.e., leave it to a report to detail)
This way we lose the current problem of how to know if a publication has been checked if a full claim can't be made, which in itself is a good thing.
We can still keep the disclaimer property if it is useful to some publishers, but what are people's opinions on including this new conformance claim?
I know that Universities are starting to write into their procurement requirements that any digital materials purchased must be WCAG-AA compliant and be 3rd party certified. So how would having this help? Two such universities which have already done this is the University of Phenix and the University of Texas, and I fully expect other universities and other educational and government agencies will require the same moving forward.
To me it sounds like those publishers who want their educational materials to be adopted at the university level must put in that metadata of WCAG-AA conformance and certifiedBy anyways which were the publishers that are most concerned with.
I think there is no harm in adding this and if those publishers don't want to put in that conformance claim will probably not get university adoption since they won't meet the procurement requirements. Once they figure that out I bet they will start putting in that WCAG-AA conformance information since technically they have passed certification and are meeting that level of conformance with only the possibility of not 100% compliance if you really dig into it as George said, if you look hard enough you will find a problem.
If this was the case every AAA certified restaurant and hotel in this country technically shouldn't be allowed to claim that because I am sure everyone has had problems with their food or hotel stay at some point. Also every website that says they are WCAG-AA compliant are also probably in violation of some criteria if you search hard enough. So why do we care about this?
The issue appears to be flexibility of reporting, not what third parties might mandate for content they accept. I'd also expect that what someone accepts for procurement purposes and what issues are in the content would be in whatever contract they have with the publisher, not solely through the inclusion of metadata. It's too unreliable given that anyone can claim anything they want, and a disclaimer would probably be unnecessary given that minor mistakes are already an understood possibility.
The issue that keeps coming up is whether there is a way for a publisher to allow a statement such as a VPAT to speak to the conformance, or lack of, instead of making a WCAG claim in the metadata. I don't see why we shouldn't allow that given that we have a base-level requirement that is exclusive of making any WCAG claim.
I expect market forces will shape the conformance claims that publishers use, but in the interests of allowing people the flexibility to choose their own paths I don't see why we can't accept both discoverability claims augmented by reports and full conformance claims offset by disclaimers.
And as I said above, requiring discoverability metadata and then having no clear way to indicate that that's the only requirement met leaves us a hole where we can't be sure if a publication has been checked at all. All you can do is make a best guess if you find some metadata present. If we plug that hole, which I believe we should for completeness, then there's automatically no way to stop a publisher from making whatever claim they prefer.
It is OK to add conformance claim for discoverable. It would be really helpful to hear from the publishers, does it help you in communicating accurate accessibility conformance claims? It is not likely to disturb existing systems, because it is an additional thing.
Using the pattern we've already established, I'd propose we add the new identifier:
EPUB-A11Y-11_DISCOVERABLE
As mentioned above, in addition to this solving the minimal case of how to identify publications that only meet the metadata requirements (so we know they've been checked), this also appears to be useful for the case of wanting to report WCAG conformance only through a separate report. In that case, you might have metadata like:
<meta property="dcterms:conformsTo" id="conf">EPUB-A11Y-11_DISCOVERABLE</meta>
<meta property="a11y:certifiedBy" id="cert" refines="#conf">Acme Certifiers</meta>
<link rel="a11y:certifierReport" refines="#cert" href="https://example.com/report/path"/>
<meta property="schema:accessibilitySummary"> ... This publication substantially conforms to WCAG 2.0 Level AA as described in the linked report.</meta>
Is this sufficient to address your needs @johnfoliot ?
@clapierre We can also keep the optional disclaimer property proposed in the existing PR if you think it will be useful for your needs. I still have to make some improvements to that.
But if this sounds workable, I'll try to get an updated pull request together.
@mattgarrish Yes I would think that having both would still be helpful. So would this mean there would be 2 conformsTo statements one with EPUB-A11Y-11_DISCOVERABLE, and the other with EPUB-A11Y-11_WCAG-21-AA or could you include both in one conformsTo statement?
Also should it just be an overarching EPUB-A11Y_DISCOVERABLE or do we need to have multiple of them for EPUB-A11Y-10_DISCOVERABLE, EPUB-A11Y-11_DISCOVERABLE, etc. as the spec evolves.
So would this mean there would be 2 conformsTo statements
No, because meeting the discoverability requirements is already implied by meeting the content conformance requirements. Those strings say you've met the full requirements of the specification (metadata and epub objectives) plus WCAG. This new string say you've only met the metadata.
Adding multiple strings might also confuse anyone processing the metadata, as it wouldn't be clear if one is in error.
Also should it just be an overarching EPUB-A11Y_DISCOVERABLE
The problem comes if the metadata requirements change in the future in an incompatible way. I'd keep these version-specific just as we keep the content conformance.
(We can't retroactively add conformance for 1.0, so we have to live with this only being available going forward.)
ha all, here are some further considerations regarding three different audiences involved that may be confused if we include a new conformance level (limited to accessibility metadata):
- end user: an end user normally does not know all the technical specifications (the differences between WCAG versions or between different levels), in most cases finding metadata related to accessibility means for the user that the content is accessible; in the case of publications complying only with metadata requirements it would not be so, generating confusion and - in fact - lowering the perceived quality of all accessible ebooks
- retailers: for some legislations (such as the European Accessibility Act) retailers must sell only accessible ebooks, it is their task to make sure that a given ebook is accessible according to the legislative requirements (with possible fines in case they sell non-accessible products); the new conformance level does not give back to the retailer any useful information about the actual accessibility of the content
- European Commission: The European Commission is analysing EPUB Accessibility 1.1 in order to propose it as a standard for accessible ebooks to be sold in Europe. The introduction of this new conformance level may confuse the Commission by allowing publishers to produce publications conforming to EPUB Accessibility 1.1, but potentially not to WCAG
best
@cmussi thanks for the comments. I was getting concerned because we were not hearing a lot from implementers. You have raised good points to think. One of the objectives of EPUB accessibility 1.1 is acceptance by EU as a technical specification, so anything that can cause a confusion on EU side is concerning.
so anything that can cause a confusion on EU side is concerning
I'm starting to think, given the time constraints, it might be better to defer this to the next revision so we have more time to figure out a solution that works for everyone. Rushing solutions always comes back to bite us and we don't seem to be getting any closer to a consensus view on this.
If you don't want to claim conformance, you can still use the certifier metadata and link a report.
I tend to agree with Matt. We do not want to create confusion. Ultimately this is a WCAG 2.x problem, and most organizations have accepted the use of WCAG 2.0 as a requirement. Deferring this to a future version, perhaps once WCAG 3 is out, would be the safer approach.
The issue was discussed in a meeting on 2021-09-23
- no resolutions were taken
View the transcript
4. Normative references to a specification of metadata properties are needed.
See github issue #1799, #1767.
See github pull request #1818, #1815.
Avneesh Singh: These are the reminders, there are some PRs with requests to review
Avneesh Singh: there's some further discussion on zero-tolerance for a11y review, please provide comments
… AOB?
George Kerscher: Asking for comments for these on GH?
Avneesh Singh: The PRs need review, and the last one is an issue, please provide comments on that
George Kerscher: Personally, the approach being taken for discovery seems ok
Avneesh Singh: There's more emails
Charles LaPierre: I was just looking at the editor's note FXL techniques
… diff and preview have errors
Charles LaPierre: Add editor's note mentioning the fxl techniques #1818
Charles LaPierre: https://github.com/w3c/epub-specs/pull/1818
Wendy Reid: It's already merged, so there's no "diff"
John Foliot: matt, FWIW, another emergent spec with a section related to security and privacy. Note that this WG are in ongoing discussions with i18n folks at W3C. See: https://w3c.github.io/personalization-semantics/content/index.html#privacy-and-security-considerations
Matt Garrish: Just look at the file changes
Avneesh Singh: Let's continue discussion on GH
… APA call times will be finalized, and I will send them to the mailing list
… see you all in the 3.3 call tomorrow
… have a nice day!
I'm going to move this issue to the CG tracker and close the open pull request. It's probably better to incubate the disclaimer property there first, and it doesn't have to be in a formal specification to be used in the package document. We can also frame the note defining it to encompass the problems we've run into and seek more feedback on how to tackle this problem in the future.