WebVTT subtitles from pipe are not generated in real-time

[Nixon197]

System info

Operating System: CentOS Linux 7 Shaka Packager Version: c1f64e5-release

Issue and steps to reproduce the problem

packager 'in=udp://127.0.0.1:22223?reuse=1,stream_selector=0,init_segment=video/video_init.mp4,segment_template=video/video_$Time$.m4s' 'in=udp://127.0.0.1:22223?reuse=1,stream_selector=1,init_segment=audio/audio_es_init.mp4,segment_template=audio/audio_es_$Time$.m4s,language=hrv' 'in=pipe1,stream=text,init_segment=sub/text_init.mp4,segment_template=sub/text_$Number$.m4s' --preserved_segments_outside_live_window="5" --io_block_size 1512 --segment_duration="6" --fragment_duration="6" --default_language=hrv --dump_stream_info --time_shift_buffer_depth 1800 --mpd_output test.mpd --minimum_update_period=3 --suggested_presentation_delay=12

Extra steps to reproduce the problem? ffmpeg -re -txt_format text -txt_page 888 -flush_packets 1 -fix_sub_duration -stream_loop -1 -i NOVA_PID-121_PAGE-888.ts -map 0:s:0 -flush_packets 1 -c:s webvtt -f webvtt pipe: > pipe1 -map 0:v -map 0:a -vcodec libx264 -b:v 1500k -acodec aac -b:a 64k -ar 48000 -af "aresample=async=1:min_hard_comp=0.100000:first_pts=0" -vbsf h264_mp4toannexb -f mpegts udp://127.0.0.1:22223

What is the expected result? Subtitles are generated in real time from ffmpeg pipe.

What happens instead?

When trying to load subtitles from pipe, subtitles are initialiy loaded only when pipe buffer is full (65K) disregarding io_block_size parameter in shaka configuration. Also when i lower io_block_size parameter bellow 1512 to get real time generation of subtitles i get "Failed to detect the container type from the buffer" error.

When initial (65k) pipe buffer is filled and subtitles detected, subtitles are generated but only when buffer reach 1512 bytes resulting in subtitles only visible in "time shift" but not in real live.

For testing purpose you can use NOVA_PID-121_PAGE-888.ts from ccextractor sample files: https://drive.google.com/open?id=0B_61ywKPmI0TYnNpYTBHRFdkRGM

If I can help in any way i will be glad to do so.

BR

[kqyang]

@Nixon197 Thanks for reporting the problem. We should use different cache size for different type of streams.

Here is one way to address the problem. Can you try if it works:

diff --git c/packager/file/threaded_io_file.cc w/packager/file/threaded_io_file.cc
index ff8ca731..15fe00cf 100644
--- c/packager/file/threaded_io_file.cc
+++ w/packager/file/threaded_io_file.cc
@@ -12,6 +12,30 @@
 #include "packager/base/threading/worker_pool.h"
 
 namespace shaka {
+namespace {
+
+constexpr char UTF8_BYTE_ORDER_MARK[] = "\xef\xbb\xbf";
+
+// The number of bytes sufficient to detect whether the file is WebVTT or XML.
+constexpr size_t kInitialReadSize = 10u;
+
+bool StartsWith(const uint8_t* buffer, size_t buffer_size, const char* prefix) {
+  size_t prefix_size = strlen(prefix);
+  return (prefix_size <= buffer_size &&
+          memcmp(buffer, prefix, prefix_size) == 0);
+}
+
+bool CheckWebVtt(const uint8_t* buffer, int buffer_size) {
+  const int offset =
+      StartsWith(buffer, buffer_size, UTF8_BYTE_ORDER_MARK) ? 3 : 0;
+  return StartsWith(buffer + offset, buffer_size - offset, "WEBVTT");
+}
+
+bool CheckXml(const uint8_t* buffer, int buffer_size) {
+  return StartsWith(buffer, buffer_size, "<?xml");
+}
+
+}  // namespace
 
 ThreadedIoFile::ThreadedIoFile(std::unique_ptr<File, FileCloser> internal_file,
                                Mode mode,
@@ -168,15 +192,26 @@ void ThreadedIoFile::RunInInputMode() {
   DCHECK(internal_file_);
   DCHECK_EQ(kInputMode, mode_);
 
+  size_t read_size = kInitialReadSize;
+  bool first_read = true;
   while (true) {
-    int64_t read_result =
-        internal_file_->Read(&io_buffer_[0], io_buffer_.size());
+    int64_t read_result = internal_file_->Read(&io_buffer_[0], read_size);
     if (read_result <= 0) {
       eof_.store(read_result == 0, std::memory_order_relaxed);
       internal_file_error_.store(read_result, std::memory_order_relaxed);
       cache_.Close();
       return;
     }
+    if (first_read) {
+      first_read = false;
+      if (CheckWebVtt(&io_buffer_[0], read_result) ||
+          CheckXml(&io_buffer_[0], read_result)) {
+        // Read the data byte by byte for text files.
+        read_size = 1u;
+      } else {
+        read_size = io_buffer_.size();
+      }
+    }
     if (cache_.Write(&io_buffer_[0], read_result) == 0) {
       return;
     }

[Nixon197]

Hi,

just tested your patch on latest master release and now i am getting following error even if I set --io_block_size 65536

[0511/103309:INFO:demuxer.cc(89)] Demuxer::Run() on file 'pipe1'. [0511/103309:INFO:demuxer.cc(155)] Initialize Demuxer for file 'pipe1'. [0511/103309:INFO:demuxer.cc(89)] Demuxer::Run() on file 'udp://127.0.0.1:22223?reuse=1'. [0511/103309:INFO:demuxer.cc(155)] Initialize Demuxer for file 'udp://127.0.0.1:22223?reuse=1'. [0511/103312:ERROR:demuxer.cc(202)] Failed to detect the container type from the buffer: 474011100042F0250001474000100000B00D0001C100000001F0002AB104B2FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF475000100002B0170001C10000E100F0001BE100F0000FE101F0002F44B99BFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF474101300140000001C000F7808005210007D861FFF14C8003FFFCDE04004C61766335382E3131312E31303000422008C11838FFF14C8001BFFC211004608C1CFFF14C8001BFFC211004608C1CFFF14C8001BFFC211004608C1CFFF14C8001BFFC211004608C1CFFF14C8001BFFC211004608C1CFFF14C8001BFFC211004 [0511/103312:VERBOSE1:status.cc(74)] 3 (INVALID_ARGUMENT): Failed to detect the container type.