Live stream with subtitles (WebVTT)

[pyahmed]

System info

Operating System: macOS Mojave 10.14.6 Shaka Packager Version: 2.3.0

Issue and steps to reproduce the problem

Source is UDP live stream, I intend to inject live subtitles. All subtitles present in the source vtt (timed into the future) show up correctly in the player. If I want to add a new subtitle I add it to the source vtt file and save it. Even if the cue is in the future, it will be ignored by packager and not show up in the stream. So far nothing new, so I read through #380 and produced a python script that adds empty cues with 100ms duration (continuously), then once in a while a real cue with text is added. Again nothing shows up.

Packager Command: packager-osx 'in=udp://192.168.0.239:40000,skip_encryption=1,stream=video,segment_template=z_video_$Number$.ts,playlist_name=z_h264.m3u8' 'in=input_text.vtt,stream=text,segment_template=z_text_$Number$.vtt,playlist_name=z_text.m3u8,hls_group_id=text,hls_name=de' --hls_master_playlist_output z_h264_master.m3u --hls_playlist_type LIVE

What is the expected result? Subtitles should be visible in the output.

What happens instead? Subtitles are not visible in the output.

Has there been any progress on this, is there any alternative way to get 'dynamically' growing vtt files as source to work? Or other way to get live subtitles injected?

[kqyang]

There are a few changes @jakubvojacek did to make it work. @jakubvojacek are you interested in sending a pull request to merge your changes to master? It can be put under a configurable flag.

[jakubvojacek]

Hello,

we didn't do anything on the packager side apart from what is listed and described by you in #380.

The main issues we're still fighting with, are in packager - like that it needs the secondish heartbeats, single timing issue in input can break the whole packaging process (one timestamp that is much more in future for whatever reason - it should disregard subs (or put to queue of some kinds) that are far in the future compared to the video etc).

Thank you Jakub

[pyahmed]

So essentially at the moment there is no way to make live subtitles work in any reliable way (DASH or HLS)?

[Canta]

Hi everyone.

I did successfully modified shaka packager in order to be able to read webvtt input from an UDP stream. In order for this to work, there are two modifications needed:

1. Shaka packager is currently too strict (in my opinion) with the webvtt parsing: if it doesn't find the WEBVTT header, it throws an error and the process ends there. I've changed that error condition to a warning condition, so the process can continue even without the header. This is key for live streams, as a constant stream of webvtt may only include the header just at the beginning, and shaka packager can be instanced any time (not in sync with webvtt's beginning) thus missing the header (even when the content is totally fine).

2. That thing with the header was important in order to auto-detect input format. But that is a problem only if you actually want to auto-detect the input format, and I don't want to do such thing: I want to be able to explicitly tell shaka packager "this stream has content in X format". So, for that, I added a input_format stream parameter, which give the ability to set explicitly the input format, and so the auto-detection does not triggers.

With ""webvtt" as explicit input format (no auto-detection), and changing the header check from error to warning (no process ending), shaka packager works fine with an UDP live webvtt text as input.

I plan to prepare a PR tomorrow bringing those changes. In the meantime, I let this comment here to wake up the issue and give some context for the future PR.