(Extended) Audio Description support

[brennanyoung]

We' re using plyr.js, and very happy with it.

Now we're starting to tackle Audio Description (for video). I'm pleased to say that I was able to bolt-on a very simple implementation using either the webSpeech API or an aria-live region with a few lines of js.

Each of these approaches has pros and cons, but I am posting here to ask whether anyone else has tried this. Also, is there any interest in adding an audio-description feature to plyr.js?

The approach I have taken is to provide a VTT file, indicating that it is a 'description' in the config json file. I was delighted to see plyr.js accepted this without complaint. After the video loads, I check to see if there are any text tracks of kind 'description'. At the moment, I just go with the first such track, although in theory there may be multiple description tracks. (Not sure what to do in that case).

I add a listener for 'cuechange' to the description track. Every time we get a new cue, I get hold of the active cue. (Again, there may be multiple active cues, but I just take the first one for now). I then pull out the text of the cue.

From here, there are two obvious options:

1. Inject the description cue text into an <output> element or aria-live region, allowing screen reader users to 'hear' the cue using their preferred synthetic voice, rate and pitch. Users without screenreaders will not notice any difference. But it would not be possible to coordinate video pause/resume with the start and stop of screenreader announcements, since there is no browser API for such.
2. Pass the description cue text to the Web Speech API and announce it. (The synth voice, rate and pitch must be decided. A deluxe solution perhaps, might offer a preferences widget to the user). The advantage of this approach is that we can automatically pause the video while the speech synth is active, and the resume play when it concludes.

Does any of this sound like it could be part of plyr.js? Or is it better to bolt this stuff on afterwards?

If these enhancements are suitable for plyr.js, I don't think it need be a significant change, but I can imagine that the config file could specify whether an aria-live region is used (e.g. supply its id, or have one generated with createElement). We might imagine the audio description will appear on screen somewhere, although it should probably look distinct from 'ordinary' captions.

The config file could (alternatively) specify some preferred synth settings (voice, rate, speed). And perhaps a scripting interface for changing these settings could be exposed. Maybe even a GUI for doing that (as with captions).

Any comments, suggestions, caveats are welcome.

[JayBox325]

Audio Description support is something I'd be interested in seeing added to Plyr! Not many of my clients have the resources to generate the descriptions, but it'd be a great option to offer to one in particular who's users would really benefit.

[missmatsuko]

Would love to be able to pass in additional audio sources for descriptive audio and dubbing tracks.

Audio descriptions are required for WCAG level AA compliance.

[brennanyoung]

Just want to mention that I have (since the post above) determined that relying on a screen reader to handle AD is not fully WCAG compliant.

There is a class of users who rely on AD, but don't use screen readers (or braille devices or equivalent 'announcer' ATs). These users will be left without AD if you don't provide an actual audio source.

Therefore, the best approach (apart from recording and editing actual AD audio) is option 2: Feed the AD cues to the web speech API. This works very well (with plyr too!), and you can quite easily implement an auto-pause, to give the AD to cue time to complete before continuing, and this complies with WCAG SC 1.2.7 - a level AAA success criterion, which is nonetheless required for level AA compliance if the AD cues overlap the spoken soundtrack.

[michaelwoodca]

Is there a way to include an AD track that can be toggled through the UI?

[brennanyoung]

Yes, you can have (say) "English" and "English with Audio Descriptions" (or whatever non-English audio you prefer) included the 'languages'. Switching AD on and off would use the same mechanism as choosing a different language.

We might regard this as a bit of a hack, but it is viable today, and the same mechanism is not unusual in commercial DVD/BluRay releases. (If anyone remembers those).

A Web Speech API implementation would require a new toggle UI control.