Investigate use of ffmpeg

[YurySolovyov]

Experiments show that it is possible to make a web server that can help us support most of ffmpeg supported audio formats. This would likely solve most of issues related to lack of native support of different codecs in chrome. Also ffmpeg provides decent metadata query feature that provides some essential info if available (loweredMetas in museeks terms).

If we choose to integrate it, everything else is implementation details and I'm sure we can reach consensus there.

So far I know that node module for ffmpeg can expose streaming interface for conversion that looks like:

ffmpeg(path).format(outputFormat); // -> ReadableStream

Where path is filepath and outputFormat is extension like wav or mp3. From what I can tell, we are interested mostly in .wav output format since it is lossless and we won't loose anything from .flac or mp3. Correct me if I'm wrong.

Another thing to consider is binary size: ffmpeg weights something about 30 megs, and I'm not sure how much of this we really need. I tried to reduce file size though build config, but no luck so far. I'm not sure if this is a big issue though.

Thoughts? Questions?

[YurySolovyov]

This may solve #53, #103, #101

[YurySolovyov]

Some status update on this:

1. it is totally doable
2. extracting metadata works, but it is more raw then one we get from out current modules, and needs more checks and edge cases support.
3. actual playback is fine, playing, switching, seeking works
4. progress rendering is broken, but that's fixable by using AudioContext (#128) and related infra.

Still todo:

• investigate how much can we reduce binary size. Needs custom builds of ffmpeg with video stuff dropped. I encourage everyone reading this to try out making custom builds and replacing binaries in my branch to test them.

[martpie]

I have a few questions for you @YurySolovyov

• Is it possible/easy to build our own FFMPEG without useless codecs like videos or images ?
• If yes, what would be the expected size of the binary ?
• Would it be possible for FFMPEG to send a stream to a node process (or from a node process, to get a ffpmeg stream) (but not play the sound) ?

[YurySolovyov]

Is it possible/easy to build our own FFMPEG without useless codecs like videos or images ?

And

If yes, what would be the expected size of the binary ?

Yes, but I can't estimate how big the impact would be. Need to investigate.

Would it be possible for FFMPEG to send a string to a node process (but not play the sound) ?

Can you explain a bit? What do you want to achieve?

[martpie]

a stream sorry*, not a string

[YurySolovyov]

Yes, this is how my demo build worked. Playing the stream or not is up to app to decide.

Maybe you can sketch up a design you have in mind? I'm curious.

[YurySolovyov]

Here is the list of possible build options, just for the record

[YurySolovyov]

@KeitIG

If yes, what would be the expected size of the binary ?

ATM, with all codecs museeks currently supports, file size is ~1.4Mb

[martpie]

That's pretty cool

[YurySolovyov]

More refs: https://github.com/PaulKinlan/paul.kinlan.me/blob/master/content/2016-12-05-ffmpeg-ideas.markdown https://paul.kinlan.me/building-ffmpeg.js/

[mainrs]

Did you consider to use upx to pack the binary?

[martpie]

Electron ships with ffmpg, there might be a way to use this version directly.

https://electronjs.org/docs/development/upgrading-chromium#verify-ffmpeg-support

[YurySolovyov]

There might be an issue where chrome only calls ffmpeg for whitelisted formats, so no matter how capable ffmpeg is, chrome might just not pass it through. This is speculation tho.