Display accessibility metadata and use it as a filter

[llemeurfr]

Add to Thorium Reader the features described in User Experience Guide for Displaying Accessibility Metadata.

• A search filter in the catalog, on the boolean "Accessible" criteria associated with WCAG A or AA (depending the level which will be finally chosen by the W3C accessibility task force).
• A11y information in the ebook info panel, starting with the "universal man" logo (visually appealing there for sighted users) with alt text "Accessible Publication", followed by the properties described in the document.

The fact that a publication is an Audiobook is already clear in Thorium, therefore we won't duplicate it with an "Audiobook" property. UI details will be defined by a designer before implementation.

Note:

• I hear that there are currently 350 such ebooks in the global "French" commercial catalog, tagged via ONIX; but I'm not sure the corresponding EPUB files have equivalent metadata.
• there are already a thousand ebooks with EPUB a11y metadata, many using WordToEPUB and generated by NGOs.

[danielweck]

The fact that a publication is an Audiobook is already clear in Thorium

Just to clarify this statement: EPUB3 Media Overlays (including ones generated / converted from DAISY full text/audio, or audio-only) are not yet identified as such in the library window / bookshelf. Thorium displays their duration / narrator (authored metadata) in the "about" modal dialog box, but these publications are not considered "audio books" per se.

[danielweck]

Additional resources: https://www.w3.org/Submission/epub-a11y/#sec-mo http://kb.daisy.org/publishing/docs/sync-media/overlays.html

[llemeurfr]

EPUB3 Media Overlays (including ones generated / converted from DAISY full text/audio, or audio-only) are not yet identified as such in the library window / bookshelf.

From what I have read, the W3C a11y document does not consider publications with media overlays as audiobooks neither.

[danielweck]

the W3C a11y document does not consider publications with media overlays as audiobooks neither

Well...

https://w3c.github.io/publ-a11y/UX-Guide-Metadata/principles/#audiobook

An indication that this publication is an audiobook which is designed to be used by listening. This designation can be applied if text will also appear on a display as long as it is not required to use the publication.

...arguably, the intended purpose of the statement "(text) is not required to use the (audio) publication" is to open the interpretation of audio + text captions / transcript (Thorium does in fact have a "captions" view for Media Overlays playback).

Now, the "audio book" / "audio-only publications" distinction is indeed useful to the end user (different UX expectations).

What sets a EPUB3 Media Overlays apart (inc. DAISY talking books) is that they usually include recorded human narration, as opposed to the "read aloud" TTS experience which relies on reading-system synthetic voices (i.e. not authored, therefore not declared / advertised in publication metadata).

The accessMode property with auditory and textual can be leveraged, see the "Displaying Accessibility Metadata" specification: https://w3c.github.io/publ-a11y/UX-Guide-Metadata/principles/#all-accessibility-metadata

Additional resources: https://www.w3.org/Submission/epub-a11y/#h-sec-disc-package http://kb.daisy.org/publishing/docs/metadata/schema.org/accessMode.html

[llemeurfr]

The trigger for the "Audiobook" property is detailed in the a11ty techniques: https://w3c.github.io/publ-a11y/UX-Guide-Metadata/techniques/epub-metadata.html#audiobook.

I don't think the EPUB3 with Media Overlays will get the meta "accessModeSufficient:auditory"; but this is something which should be made clear. Let's bring your comment to the attention of the EPUB3 WG a11Y task force. I'll do it.

And in fact, because the Audiobook property is a direct mapping on EPUB3 metadata, I agree we should display it when the conditions are met.

[danielweck]

Indeed:

<meta property="schema:accessModeSufficient">auditory</meta>
<meta property="schema:accessModeSufficient">textual, auditory</meta>

...can in fact be included in a EPUB3 Media Overlays, but most publishers / content creators will probably just include:

<meta property="schema:accessModeSufficient">textual, auditory</meta>

[danielweck]

Also note: https://github.com/w3c/publ-a11y/issues/28

[llemeurfr]

Not sure at all: "textual, auditory" means both textual AND auditory abilities are necessary to access the data (in the Audiobook section).

[danielweck]

Not sure at all: "textual, auditory" means both textual AND auditory abilities are necessary to access the data (in the Audiobook section).

I'm not sure either. The spec. says:

There may also be other accessModeSufficient entries with combinations of access modes such as "textual, visual" which indicate other ways to read the book.

[danielweck]

Links to Laurent's mailing list messages: https://lists.w3.org/Archives/Public/public-publishingcg/2021Jan/0010.html https://lists.w3.org/Archives/Public/public-epub3/2021Jan/0009.html

[ManfredMuchenberger]

In my opinion, in a fulltext/fullaudio book with human narration using media overlays we should use these 3 entries in the metadata:

meta property="schema:accessModeSufficient">textual</meta meta property="schema:accessModeSufficient">visual</meta meta property="schema:accessModeSufficient">auditory</meta

.... at least this is how I interpret http://idpf.org/epub/a11y/techniques/#meta-002

[danielweck]

Hello Manfred, I would have thought that the default accessModeSufficient should be textual,auditory, with additional accessModeSufficient properties set to auditory and textual (visual can be added too if there is a need for graphics).

[ManfredMuchenberger]

well I am by no means an expert but would be happy to know how before we start distributing these books ;-)

from this Example: Since the Author has also included descriptions for all the images, she can also indicate that a purely textual access mode is sufficient to read the content: textual Without this metadata, users would not have known that they could read the publication only via its text content.

I thought that if the full text is there, that"textual" needs to stand as a separate entry. I also thought that if the full content is available as audio, that "auditory" needs to stand as a separate entry. I also thought that if you can read the full text an screen, that "visual" needs to stand as a separate entry.

[clapierre]

I asked Madeleine Rothberg to weigh in here as she is the leading expert regarding accessibility metadata.

[madeleinerothberg]

This is confusing, and most people have some part of this correct, but not all of it. I hope this is helpful. Let me know if I should address other use cases here.

Full text, full audio book: A book that contains its entire content in both text and audio (whether that audio is pre-recorded human voice or pre-recorded text to speech) would have the following metadata:

<meta property="schema:accessModeSufficient">auditory<meta>
<meta property="schema:accessModeSufficient">textual<meta>

In addition, it could have the following line as well, which would indicate that a user could listen to the audio and look at the text simultaneously: <meta property="schema:accessModeSufficient">textual, auditory</meta>

Unless there are significant images in the book, there is no need to reference visual access modes. It is not relevant whether the user uses their eyes to look at the text, uses a screen reader to convert it to audio at the time of reading, or prints it on a braille printer and reads it that way. The resource itself is textual.

[llemeurfr]

@madeleinerothberg thanks for the information.

This implies that EPUB with media overlays will get the mention Screen Reader Friendly: yes Audiobook: yes

A confusing indication is that "W3C Audiobook" is a standard based on another format than EPUB. But the culprit is certainly the generic name given to a specific format.

In such a case (EPUB with media overlays), shouldn't schema:accessMode be set also (to 'textual', then 'auditory')? It does not add anything to the UX, but may used in another scope (?)

Note that I see all these metadata values stacked in https://w3c.github.io/publ-a11y/UX-Guide-Metadata/techniques/epub-metadata.html#example-9.1-all-metadata-fields-present.

[clapierre]

We were only discussing accessModeSufficient but yes additionally for accessMode both textual and auditory would be included if it has both text and audio within the EPUB.

[llemeurfr]

Note: in the Display Techniques for EPUB a11y metadata, "Audiobook: yes" has been replaced by "Full Audio: yes".

[danielweck]

Closing old issue https://github.com/edrlab/thorium-reader/issues/669

https://github.com/w3c/publ-a11y/tree/master/UX-Guide-Metadata

=>

https://w3c.github.io/publ-a11y/UX-Guide-Metadata/principles/

and

https://w3c.github.io/publ-a11y/UX-Guide-Metadata/techniques/

[danielweck]

Related (webpub-manifest JSON): https://github.com/readium/webpub-manifest/pull/85

[danielweck]

Parser bug: https://github.com/readium/r2-shared-js/issues/37

[gautierchomel]

This proposal follows the issue in Thorium-reader github: Display accessibility metadata and use it as a filter.

The subject is covered by accessibility task force of publishing community group at W3C and actually under active feedback from implementation projects in France, Canada, Italy at least. See references section for more context.

This implementation proposal intent to be as complete as possible to help dealing with any recommandation change in the coming years.

1. If possible stretch Tag space to one or two lines. Or place it under the book action menu (read, delete, download).
2. Add h2 level titles in a details/summary: Book details, Reading modes, Functionalities, Accessibility Information, Hazard. A code exemple in French and A code exemple in english are proposed.
3. populate the inside of details as unordered lists (ul / li) following the mapping provided below.

filters

• reading mode
  • Display transformability;
  • Fixed-layout;
  • TTS & Screen reader friendly;
  • Text & audio synchronised
• accessibility information available

This may imply that reading mode and accessibility information available are displayed in the table of list view (All books).

Another option may be to use the existing Format column and populate it with : fileFormat + readinGmode + accessibilityInfo

Alternatively a Man of Vitruve pictogram could be affected to any file claiming conforms To WCAG AA:

Mapping

Readium Controled vocabulary : schema/a11y.schema.json

Book details

Elements actually displayed by thorium (book details) are not mapped but kept here for reference.

DisplayedEN	ReadiumJSON	fileMeta	ifNotPresent
Format: EPUB			ignore
Protection: LCP			ignore
Published on 2020 by SNE			ignore
ISBN 9780000000000			ignore
Language: English (en)			ignore

Reading modes

If none of the information is provided, display : No reading mode information available. instead of Reading modes.

DisplayedEN	ReadiumJSON	fileMeta	ifNotPresent
Display transformability	"feature":" context.jsonld#L21 content="displayTransformability"	schema.org:accessibilityFeature content="displayTransformability"	ignore
Fixed-layout		no existing value in schema.org. may be extracted from format (pdf + epub3FXL In EPUB3 opf : rendition:layout property set to pre-paginated	ignore
Contents: text, audio, tactil, etc.	"accessMode":" context.jsonld#L19 content="visual", "textual", "auditory", "tactil", etc. (one line per value or coma separated values in one field)	schema.org:accessMode	ignore
TTS & Screen reader friendly	"accessModeSufficient" context.jsonld#L20: textual	accessModeSufficient: textual	ignore
Contenu audio intégral (présentation, index, notes)	"accessModeSufficient" context.jsonld#L20: auditory	schema.org:accessModeSufficient:auditory	ignore
Text & audio synchronised	"feature": context.jsonld#L21 content="synchronizedAudioText"	schema.org:accessibilityFeature schema.org:sychronizedAudioText	ignore

Functionalities

If none of the information is provided: do not display.

DisplayedEN	ReadiumJSON	fileMeta	ifNotPresent
Table of content	"feature": context.jsonld#L21 content="tableOfContents"	schema.org:accessibilityFeature schema.org:tableOfContents	ignore
Printed book pagination reference	"feature": context.jsonld#L21 content="printPageNumbers"	schema.org:accessibilityFeature schema.org:printPageNumbers	ignore

Accessibility

If none of the information is provided, display : No accessibility information available instead of Accessibility informations.

DisplayedEN	ReadiumJSON	fileMeta	ifNotPresent
any text ... lorem ipsum ... [display content as it is, mark it with publication language lang tag - this item is not in a list element but in a p]	"accessibilitySummary": context.jsonld#L18 content="any text ... lorem ipsum ..."	schema.org:accessibilitySummary	ignore
Structured	"feature": context.jsonld#L21 content="structural"	schema:structural	ignore
Logical reading order	"feature": context.jsonld#L21 content="NavigationreadingOrder"	schema:NavigationreadingOrder	ignore
Fully described graphical ressources	"feature": context.jsonld#L21 content="longDescription"	schema.org:accessibilityFeature schema.org:longDescription	ignore
conformsTo : readable human text or URL [display content as it is, mark it with publication language] lang tag	"conformsTo": context.jsonld#L7 content="readable human text or URL"	[dc:conformsTo](http://purl.org/dc/terms/conformsTo"
Conformance by: x20xxx [display content as it is, mark it with publication language]		a11y:certifiedBy	ignore
Conformance report [is a link]		dc:certifierReport	ignore
Publisher accessibility policy [is a link]		(onix only)	ignore
Button: Other informations available *open list of any schema.org:accessibilityFeature not yet parsed. Get full controlled vocabulary in locales (en.js, etc.) to be nammed later	"feature": context.jsonld#L21 content=any controlled vocabulary	schema.org:accessibilityFeature schema.org:accessibilityFeature:controlled vocabulary	ignore

Hazards

If none of the information is provided: do not display.

DisplayedEN	ReadiumJSON	fileMeta	ifNotPresent
any of the following values: flashing, motion simulation, sound, no flashing, no motion simulation, no sound, none, unknown.	"hazard": context.jsonld#L22 content=any of the following values: flashing, motion simulation, sound, no flashing, no motion simulation, no sound, none, unknown.	schema.org:accessibilityHazard	ignore

References

• User Experience Guide for Displaying Accessibility Metadata 1.0
• French prototype for book sellers and libraries
• Display Techniques for EPUB Accessibility Metadata 1.0
• partial Mapping with ONIX & MARC available in french.

[llemeurfr]

This is a great start. My first comments:

We should make so that the UI model of this modal window is not totally modified by the addition of accessibility metadata. In practice:

• The tags can be moved to the bottom of the page, no need to move them to the left (there are other actions there, like "Loan", "Download" ...). Adding a tag can be moved to the right of the last existing tag (often 1 only, when not 0), to save space.
• I agree that the Description must be on top, before the other metadata.
• currently, we have then a "More Info" section with "Published on", "Publisher" and "Language". "Format" should go on top of these.
• There is a question to solve about a generic metadata like "ISBN". It is important on a bookstore, maybe not in a reading app.
• Currently, we have a specific "LCP" section which contains property rights ("Start", "End", "Copied ...", "Printed Pages". I'm tempted to keep this section. Maybe "Start" / "End" can be presented side by side (for saving some place).
• "Reading Modes", "Functionalities", and Accessibility Information" can be presented as proposed, using a "details" html tag that displays them on-demand.
• The famous "Accessibility Summary" is missing in the proposal, let's add it as a last details tag.
• Thorium currently presents a "Progression" section (the ":" sign must be removed after the title). We will keep this section after the "Tags" section.

[llemeurfr]

Regarding filters, the table handles filters via hyperlinks, each column can support one filter only. The new filters must follow this model. The previous proposal does not allow that.

We could treat "Accessibility Information available" (this label is too long) as a new column, as it is simply a boolean. "True" may be display as an image (the Vitruve man?) or a text (which should be short).

For filters corresponding to reading modes (Display transformability; Fixed-layout; TTS & Screen reader friendly; Text & audio synchronised) we have a first question to solve: could we consider that choosing one only is enough (i.e. a user is not angry if he cannot choose Fixed-layout + TTS & Screen reader friendly. In this case, a single column would suffice. If not we have a UX issue to solve.

[gautierchomel]

Adjusted screenshots with Laurent's points. I also added a folded view so we can figure out how it would look like.

The AccessibilitySummary is shown under the Accessibility Information Detail, it's a <p> and not included into a list. The recent changes on the recommendations about this Human written piece of metadata makes that we will see very strange cases on the next years including some redundancies of information (Conforms to is an exemple).

[gautierchomel]

Also to have in mind, this addition makes a lot of elements to localize.

[gautierchomel]

Edit after team call, relevant points listed here:

• Fully accessible is redundant with Screen reader friendly
• ISBN is still a discussion (what about an identifier which is not an ISBN?)
• there is some challenges to make a phrase with 2 different meta (published on ... by ...)
• : should be suppressed (bold/non bold is sufficient differentiation)
• A flexible two columns layout maybe explored for large screen display

reviewed proposal

This proposal intends to simplify access to key information displaying it outside of a details box. Others information are still boxed, but redundancies are not exposed anymore. more details on how to fill information is provided below.

Book details

Elements actually displayed by thorium (book details) are not mapped but kept here for reference.

DisplayedEN	ReadiumJSON	fileMeta	ifNotPresent
Format: EPUB
Protection: LCP
Published on 2020
Published by SNE
Language: French (fr)

Accessibility

This is a title (h level to be determined) Then if presents the following information are displayed:

DisplayedEN	ReadiumJSON	fileMeta	ifNotPresent
TTS & Screen reader friendly	"accessModeSufficient" context.jsonld#L20: textual	accessModeSufficient: textual	ignore
Adaptable text and layout	"feature":" context.jsonld#L21 content="displayTransformability"	schema.org:accessibilityFeature content="displayTransformability"	ignore
Fixed-layout		no existing value in schema.org. may be extracted from format (pdf + epub3FXL In EPUB3 opf : rendition:layout property set to pre-paginated	ignore
Text & audio synchronised	"feature": context.jsonld#L21 content="synchronizedAudioText"	schema.org:accessibilityFeature schema.org:sychronizedAudioText	ignore
Printed book pagination reference	"feature": context.jsonld#L21 content="printPageNumbers"	schema.org:accessibilityFeature schema.org:printPageNumbers	ignore

If any Hazard information set to true, it should be displayed here too.

DisplayedEN	ReadiumJSON	fileMeta	ifNotPresent
any of the following values: flashing, motion simulation, sound, no flashing, no motion simulation, no sound, none, unknown.	"hazard": context.jsonld#L22 content=any of the following values: flashing, motion simulation, sound, no flashing, no motion simulation, no sound, none, unknown.	schema.org:accessibilityHazard [Controled vocabulary]	ignore

More about accessibility

This is a details element, summary is More about accessibility

If none of the information is provided, display into a <p>: No accessibility information available instead.

To avoid redundancies the following are not implemeted:

• Certified By (this info is found in the certfier report)
• schema:structural & schema:NavigationreadingOrder are implied by WCAG A

accessibilitySummary is now at the very end of this section.

DisplayedEN	ReadiumJSON	fileMeta	ifNotPresent
Fully described graphical ressources	"feature": context.jsonld#L21 content="longDescription"	schema.org:accessibilityFeature schema.org:longDescription	ignore
conformsTo : readable human text or URL [display content as it is, mark it with publication language] lang tag	"conformsTo": context.jsonld#L7 content="readable human text or URL"	[dc:conformsTo](http://purl.org/dc/terms/conformsTo"
Conformance report [is a link]		dc:certifierReport	ignore
Publisher accessibility policy [is a link]		(onix only)	ignore
any text ... lorem ipsum ... [display content as it is, mark it with publication language lang tag - this item is not in a list element but in a p. Last element to be displayed]	"accessibilitySummary": context.jsonld#L18 content="any text ... lorem ipsum ..."	schema.org:accessibilitySummary	ignore

Search filters

This part can be added later once some iterations have been done on the Book information panel.

Add a column titled Accessibility in the All books view. Populate it with one of the following information if present. It should be looking like a bloc (like tag).

• screen reader friendly
• Adaptable text and layout
• Audio and text synchronized

[gautierchomel]

Because conformsTo URLs leads to technical specs, it is of no use for users. Here is a mapping URL to understandable text :

URL	Text
http://www.idpf.org/epub/a11y/accessibility-20170105.html#wcag-a	EPUB Accessibility 1.0 - WCAG 2.0 Level A
http://www.idpf.org/epub/a11y/accessibility-20170105.html#wcag-aa	EPUB Accessibility 1.0 - WCAG 2.0 Level AA
http://www.idpf.org/epub/a11y/accessibility-20170105.html#wcag-aaa	EPUB Accessibility 1.0 - WCAG 2.0 Level AAA

[danielweck]

Right, but note that ConformsTo (the data model property) can be parsed from EPUB OPF XML meta or link. In the latter case (spec. v1.0) it was expected to be a URL / URI. In the former case (spec. v1.1) it may be the WCAG2.0 or WCAG2.1 strings. Thorium only reads the ConformsTo data model property, it has no idea that the value may originate from meta or link. So we need to detect HTTP URL/URI and fallback to plain text.

[panaC]

Right, but note that ConformsTo (the data model property) can be parsed from EPUB OPF XML meta or link. In the latter case (spec. v1.0) it was expected to be a URL / URI. In the former case (spec. v1.1) it may be the WCAG2.0 or WCAG2.1 strings. Thorium only reads the ConformsTo data model property, it has no idea that the value may originate from meta or link. So we need to detect HTTP URL/URI and fallback to plain text.

https://github.com/edrlab/thorium-reader/blob/4cdd2d13f3e00d8d834a3eb6f07571d089312e5b/src/renderer/common/components/dialog/publicationInfos/publicationInfoA11y.tsx#L200-L212