Progressive transcoding to start streaming immediately

[OverkillGuy]

I'd love to be able to make transcoding a file into a chunked process, so that we don't have to wait for the file to be completely transcoded to start streaming. This would improve usability on big files like movies.

I know VLC has this feature on Android (and maybe even in their 3.0 release desktop), but I suspect they reimplemented completely the Chromecast controls to be able to do it.

Required steps

1. Check the chromecast mediacontrols API for direct support of chunked streaming
2. Change calls to ffmpeg to only transcode chunks of X seconds (one minute sounds like a good first pass)
3. Update the scrubbing system accordingly and any other API.
4. PR and enjoy fame and fortune

I would naturally love to help on that new feature, I'm not just complaining here :) Any thoughts on this ?

[keredson]

agreed!

i found this:

ffmpeg -i input.mp4 -c copy -map 0 -segment_time 8 -f segment output%03d.mp4

from here.

which looks very promising.

[keredson]

no idea if the default media receiver supports any sort of list of urls tho.

[keredson]

some history here... i built this w/ progressive encoding in mind. see the wait_for_byte function here. sadly the chromecast makes two requests, one for the beginning of the file, one right at the end, to read the MOOV metadata blob. this is why you can start the file playing before the convert is done and it doesn't error, chromecast just is stuck in the "loading" mode until that last blob is written. (and then it starts playing.)

ffmpeg supports moving the MOOV to the beginning of the file, but sadly this requires a two pass conversion which is longer, and still doesn't happen until the very end.

chromecast also supports webm, which i tested and worked well for progressive encoding, but was slow as dirt. (0.1x speed on my XPS13.) ffmpeg has options to force 1x speed at reduced quality, but it still worked the laptop hard. and scrubbing ahead you still had to wait for it to catch up. given the lack of native webm content out there meaning everything would have to be transcoded, i gave up on this. (VLC uses the webm method btw.)

[fn-ix]

I've been using the Videostream (https://getvideostream.com/) Chrome extension for a while; not sure what's going on under the hood, but it has played every format I've thrown at it while starting to stream immediately and I've never experienced any lag or significant resource usage.

[rgegriff]

I want to try and take a crack at this when I get a few free weekends. It's definitely something that I have been wanting to play with. I found https://developers.google.com/cast/docs/mpl/streaming_protocols which describes using the CAF to do streaming, segmented media with HTTP Live Streaming. Definitely more complex than just using the default media receiver and throwing a file url at it, but it would totally be worth it. I am hoping I can let github pages act as the "server" for the app and serve the app and manifest file from the gnomecast instance

[keredson]

i'm all for that! i have a server that could host the app as well.

[keredson]

i have a proof of concept working for streaming without transcoding away from h264/5!!!!!

[keredson]

well maybe i spoke too soon. HLS (HTTP Live Streaming) does work, but there's an annoying sub-second pause between sections. plus it seems to keep the HUD permanently on.

https://github.com/keredson/gnomecast/tree/http-live-streaming if anyone wants to play w/ it.

[keredson]

fixed pause. fixed broken seeking. hud issue seems to have been a bad crashed state on my chromecast - rebooting fixed.

[horiac7]

Hello!

Have you tried directing ffmpeg output to a pipe and then reading from there and serving?

I found a decent implementation in this repository and file (have not tested this personally): Mkchromecast : https://github.com/muammar/mkchromecast/blob/master/mkchromecast/video.py

Please have a look, I can also highlight the interesting parts below: Setting up the ffmpeg command to pipe:

if input_file is not None and subtitles is None and mkv is False:
            # Command taken from
            # https://trac.ffmpeg.org/wiki/EncodingForStreamingSites#Streamingafile
            command = [
                "ffmpeg",
                "-re",
                "-i",
                input_file,
                "-map_chapters",
                "-1",
                "-vcodec",
                "libx264",
                "-preset",
                "veryfast",
                "-tune",
                "zerolatency",
                "-maxrate",
                "10000k",
                "-bufsize",
                "20000k",
                "-pix_fmt",
                "yuv420p",
                "-g",
                "60",
                # '-b', '900k',
                "-f",
                "mp4",
                "-movflags",
                "frag_keyframe+empty_moov",
                "pipe:1",
            ]

I guess these -movflags frag_keyframe+empty_moov options work better than what you mentioned above about having to have a 2-pass.

And then, serving this:

@app.route("/" + appendtourl)
def stream():
    process = Popen(command, stdout=PIPE, bufsize=-1)
    read_chunk = partial(os.read, process.stdout.fileno(), chunk_size)
    return Response(iter(read_chunk, b""), mimetype=mtype)

[horiac7]

Forget my previous comment, in the mean-time I have experimented a bit with the above technique without any success - I think that it might not be the best due to lack of information about video length and not beeing seekable. Excuse my ignorance of not actually testing the above before posting it, but I will leave it there for comparison sake.

I want to report on another small investigation I started, about how Videostream for Chromecast works (the downloadable one, not the Chrome extension)

When selecting a video for casting, I saw an ffmpeg command issued (as you can observe, it got installed at the following path) with the following commandline: C:\Users\horiac\AppData\Local\Videostream\app-0.3.6\videostream-native\ffmpeg.exe -loglevel panic -hide_banner -nostats -copyts -y -dump_attachment:t "" -i E:\movies\redacted\redacted.mkv -map 0:0 -vcodec libx264 -preset veryfast -threads 4 -pix_fmt yuv420p -vprofile high -vlevel 4.1 -g 48 -keyint_min 48 -sc_threshold 0 -refs 1 -vbsf h264_mp4toannexb -vf scale=s=1280x720:flags=bilinear -vb 1026k -maxrate 1026k -minrate 1026k -bufsize 1026k -map 0:1 -acodec aac -ac 2 -ar 48000 -ab 192k -strict -2 -flags +cgop-global_header -muxdelay 0 -muxpreload 0 -f ssegment -segment_time 4 -segment_format mpegts -segment_list C:\Users\horiac\AppData\Local\Temp\videostream-native\chunks-1576691686474\chunks.csv -segment_list_type csv -segment_start_number 0 -segment_list_flags +live C:\Users\horiac\AppData\Local\Temp\videostream-native\chunks-1576691686474\c%010d.ts

Seems to be using the streaming segment format - I don't know the differences between this and -f segment that you have used in the http-live-streaming branch.

Then I checked the contents of the chunks folder:

> dir

 Directory of C:\Users\horiac\AppData\Local\Temp\videostream-native\chunks-1576691686474

18.12.2019  19:57    <DIR>          .
18.12.2019  19:57    <DIR>          ..
18.12.2019  19:56           757,264 c0000000040.ts
...
18.12.2019  19:57           247,596 c0000000050.ts
18.12.2019  19:57             1,796 chunks.csv
              12 File(s)      7,125,116 bytes
               2 Dir(s)  45,666,041,856 bytes free

where I could see a growing list of *.ts files as the stream progressed. The older ones were deleted after some time, and the chunks.csv kept information about each individual duration of the ts files:

c0000000000.ts,0.000000,4.087422
c0000000001.ts,4.087422,8.091422
c0000000002.ts,8.091422,12.095422
...
c0000000056.ts,224.307422,228.311422
c0000000057.ts,228.311422,232.315422

When seeking, a new chunks folder got created and the process of creating chunks 'restarted' - I believe a new ffmpeg command was submitted for this.

I hope that this information may help you with this task.

[keredson]

The http-live-streaming branch mostly works. The only hiccup is 5.1 audio, which I couldn't get to pass through.

[uninhm]

Why on vlc (Android) is not necessary to transcode?

[jlost]

Is the 5.1 audio passthrough needed? Would it make sense to merge your branch?