cookbook-recipes icon indicating copy to clipboard operation
cookbook-recipes copied to clipboard

Audio recipe with captions

Open glenrobson opened this issue 2 years ago • 5 comments

Recipe Name

Very similar to: https://iiif.io/api/cookbook/recipe/0219-using-caption-file/ but with an audio file.

Use case

glenrobson avatar Sep 01 '23 16:09 glenrobson

@glenrobson Captions aren't a thing for audio only files, instead we have transcripts. Does the updated #253 for transcripts cover this use case so we can close this? https://github.com/IIIF/cookbook-recipes/issues/253

elynema avatar Mar 17 '25 16:03 elynema

A lot of places use captions for audio only files. It isn't accessible to not display them, since accessibility issues are a spectrum. Stanford uses the video player since the videojs audio player doesn't work as well for captions, but I think this is an accessibility issue if you aren't providing captions for audio files.

dnoneill avatar Mar 17 '25 17:03 dnoneill

@dnoneill Do you have an example of captions being displayed for an audio only item? That would be great to see. Also helpful if you have a good link to explain why captions (as separate from transcripts) are required for audio only files so I can increase my understanding.

So far, I've been looking closely at the Understanding Guideline 1.2 for Time-Based Media page from W3C. I feel like it indicates that captions are required for synchronized video/audio material, and an "alternative for time-based media" is required for audio only (or video only), which includes a descriptive transcript (but would be audio description for a silent movie, etc).

From the IIIF perspective, we've been taking the stance that captions are identified as text to be rendered within the media player itself over top of video and transcripts are identified as text to be rendered alongside the media. It would be helpful to know if we need to adjust that.

elynema avatar Mar 17 '25 18:03 elynema

We aren't using IIIF for these yet but here is an example of an audio file with captions https://purl.stanford.edu/kd832tc8057. So my thought process is some people might be hard of hearing, or have trouble with certain words being spoken so having both the audio they can listen to and the captions to confirm their understanding is good accessibility. Additionally, I believe some neurodivergent individuals find captions as an aid for understanding audio. W3 says it isn't a requirement for WCAG but it is a good accessibility https://www.w3.org/WAI/media/av/captions/#checklist. So while it isn't a requirement it is good accessibility.

dnoneill avatar Mar 17 '25 18:03 dnoneill

@dnoneill Thanks for sharing. It looks like W3C is assuming that transcripts are not synchronized with audio using a timed text format like WebVTT. I thought it was more common for media players to provide a way to display a synchronized transcript for audio only materials (which seems to be the same thing as a caption) than for media players to provide a way to display captions over top of a still image for audio only files. I may be mistaken on that.

However, you are indicating a clear use case for captions for audio only files, so I agree this should be retained.

elynema avatar Mar 17 '25 19:03 elynema