Embedded VTT captions not showing up during playback for audio-only HLS stream

[elohanlon]

Description

This GitHub issue is half bug report and half documentation/demo update request:

When I point VideoJS at a video+audio HLS stream that has embedded VTT captions, the "CC" button appears in the player and I'm able to select the caption track. Then the captions are displayed as the video plays. All good there.

By contrast, when I point VideoJS at an audio only HLS stream that has embedded VTT captions, the "CC" button appears in the player and I'm able to select the caption track. However, the captions are not displaying as the audio plays.

In the above two scenarios, the backing VTT files are identical. And the audio file is an audio-only m4a version of the source mp4 video file.

On the VideoJS demo page (https://videojs-http-streaming.netlify.app), which I've seen referenced in other GitHub issues, there are several examples of audio HLS playback, but none of them have captions. Would it be possible for the maintainers to add an audio + embedded VTT HLS captions example? I think there may be a bug in VideoJS, which is causing the caption display problem. An example on the demo page would reveal whether or not this is the case for anyone other than me, and would also be generally helpful for anyone who wants to see an example of an audio HLS stream with embedded VTT captions.

Interestingly enough, even though the audio-only embedded HLS caption scenario isn't working for me, I am able to get captions to show up during playback if I point VideoJS to an audio-only HLS stream and include a html element inside the <audio> element (or <video> element -- I've tried both) with a src value pointing to a separate, external caption file.

My HLS streaming server is behind authentication, so I don't currently have an easy way of creating my own public demo for this GitHub issue, so I'm hoping that someone on the VideoJS side can more easily update the demo on the https://videojs-http-streaming.netlify.app site. And in case it's relevant, I'm using the Wowza Streaming Engine to stream.

One last note here: Why do I want to show captions for an audio file? There are a few reasons:

• It's helpful for listening to recordings in which the audio quality isn't so good, and reading real-time text can improve the experience.
• It's good for accessibility, to support users who have difficulty hearing.
• It's helpful for users who want to listen to audio at high speed, so they can visually catch words that were missed.

Thank you for any help you can offer here!

Reduced test case

No response

Steps to reproduce

1. Point VideoJS at an audio only HLS stream that has embedded VTT captions.
2. Click on the "CC" button appears in the player, and select the caption track (which confirms that the track is present and recognized).
3. Play the audio file in the player, but observe that captions are not displaying as the audio plays.

Errors

No errors in the player or console.

What version of Video.js are you using?

7.21.1

Video.js plugins used.

No response

What browser(s) including version(s) does this occur with?

Chrome 108

What OS(es) and version(s) does this occur with?

macOS 12.6.1

[welcome[bot]]

👋 Thanks for opening your first issue here! 👋

If you're reporting a 🐞 bug, please make sure you include steps to reproduce it. We get a lot of issues on this repo, so please be patient and we will get back to you as soon as we can. To help make it easier for us to investigate your issue, please follow the contributing guidelines.

[TheCelavi]

I can confirm same behaviour as well.

[OwenEdwards]

@TheCelavi I know that for HTML media elements, the <audio> element doesn't support VTT captions:

The

https://developer.mozilla.org/en-US/docs/Web/HTML/Element/audio#accessibility_concerns

The problem for an audio player is that it doesn't have any region on the page to render the captions into - there's no video frame or poster image over which captions can be displayed.

But I don't know about HLS. Does video.js even create a display region (separate from the controls) when it encounters an audio-only HLS stream? Can a poster be specified which might create that region on the page for captions?

Elsewhere we have recommended using a <video> element (with video.js taking over the video playback) for audio-only content with VTT captions, as the MDN page mentions. But I don't know how you would do that with HLS if specifying a poster doesn't change the behavior.

[TheCelavi]

@OwenEdwards thank you.

[gkatsev]

Video.js can have a display area for audio, in fact that's the default. There are two options for this as well. audioOnlyMode for when you only want the control bar and audioPosterMode for when you want the poster to remain visible. In the default or audioPosterMode, there is a place to display the captions.

How are the embedded captions for you? Is it a segment vtt playlist? A sample stream would be helpful to diagnose the issues.

[TheCelavi]

@gkatsev I have realised later on that if subtitles are provided within .m3u8 file, they MUST be segmented (which is really unnecessary, at least for video/audio on demand). Either way, will have to deal with that sometime... thanks!