gioui-plugins
Video Player
Why that plugin should be added? Video/Music players are very common on websites and apps. I think multiple apps would like to play videos or even streaming videos from external source.
What features this plugin should have and how it should work?
It should work similar to "Webviewer", or better. The idea is to have mediaplayer.RectOp{} which defines the area to show the content and fill it with the video.
What alternatives you've considered? None. It's possible to create some bridge between Golang and FFmpeg, but that requires FFmpeg to be installed or bundled with the app (which conflicts with GPL license from FFmpeg).
Is this plugin implementable on all OSes (Android, iOS, macOS, WASM, Windows...)? No.
Did you known any particular limitation for specific OS?
On WASM it's not possible to directly access video codecs, it would require WebCodecs API, which is only supported by Chromium. It could fallback to <video>, but makes customization harder. Also, the video-codecs depends on the system itself, not only OS. It's possible to have one Windows machine without AV1, and another one with AV1 codec. On Andriod/iOS/macOS that is more difficult.
Handling the video-codec itself (without external system dependency) is VERY hard, and also lacks hardware-acceleration (such as GPU).
Yeah i also think a webview is the way to go. Let the browser do the work.
Do you know if the gio webview can be placed within a layout yet ?
Btw thanks for raising this one. Have been hitting my head against brick wall on video :)
I used to use this too: https://github.com/go-flutter-desktop/plugins/blob/master/video_player/plugin.go its pretty the golang code for extracting the frames so that the GIO canvas can use them.
The issue is that it uses ffmpeg, which is something that I'm trying to avoid:
It's possible to create some bridge between Golang and FFmpeg, but that requires FFmpeg to be installed or bundled with the app (which conflicts with GPL license from FFmpeg).
The alternative seems to use the codec directly, most OSes (windows, macos, ios, android) offers hardware-acceleration, which by luck should make possible to decode videos without ffpmeg. Even JS have Codec API now (not sure how to use it).
Displaying each frame is another issue, since Gio doesn't have a way to render textures directly (far I know) and external "views" aren't supported either (the WebView is already clunky because of that). It works, but...
Yeah I agree. FFmpeg is just the wrong fit. Fine for a server but not on a client.
Regarding native OS codec, for example, for IOS and Mac there is https://github.com/progrium/macdriver/tree/main/macos/avfoundation. It might be a decent reference, but I have not tried this.
There is a GIO video and audio player at least.
https://github.com/mearaj/gio-video. Nice stuff @mearaj
Its using ffmpeg and the reisen lib to break the video into frames for gio.
The other aspect of accessing native Audio and Video API for each target OS will be that it will also make it easy to do a Camera plugin to display the Video and Audio with the same gio video player that @ mearaj has started off.
Hey
@inkeliz @mearaj
I have not tried it yet, but https://github.com/arthenica/ffmpeg-kit looks like it wil give us what we want with regarding to video on Mobile and Desktop.