godot
godot copied to clipboard
godotengine
[3.x] Support WebM seeking operation
Support webm video seeking operation with minimal modification. Currently, it only support 3.x version since master 4.0 version has abandoned webm format support.
Modified files:
1.modules/webm/video_stream_webm.cpp
2.thirdparty/libsimplewebm/WebMDemuxer.cpp
3.thirdparty/libsimplewebm/WebMDemuxer.hpp
https://user-images.githubusercontent.com/99071212/152667074-e226a8f6-b3d7-4ef0-b5af-31bec11eeb61.mp4
see #14430
Thanks for your feedback, since I'm not an expert in video decoding, but I'll try my best to fix this issue.
see #14430
@jamie-pate i have fix the issue, and please verify it on 1ede4ca4bdc5ae39081bb199541594054fbda161. Thanks!
@John-Gdi nice work！ Do you know how to fix VideoStreamWebm playing slowly in Html5? I really want to know where the problem is.
@John-Gdi nice work！ Do you know how to fix VideoStreamWebm playing slowly in Html5? I really want to know where the problem is.
Video decoding will always perform poorly in HTML5 for two reasons:
- yasm assembly optimizations cannot be used.
- Decoding is performed on the CPU, via a pure C WebAssembly layer, which is slow.
There isn't a good way to work around this, as WebAssembly cannot make use of the browser's native video decoding functionality (which would allow for hardware-accelerated decoding). Theora might perform better as it's less demanding on the CPU, but I wouldn't expect it to be fantastic there either.
@John-Gdi nice work！ Do you know how to fix VideoStreamWebm playing slowly in Html5? I really want to know where the problem is.
You would likely need a whole new way to play video in the html version.. using the built in video element to create a texture for webgl (like this https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API/Tutorial/Animating_textures_in_WebGL ). godot's OS.native_video_play https://docs.godotengine.org/en/stable/classes/class_os.html#class-os-method-native-video-play call would be another way, but docs say they are only currently implemented for ios
You could possibly just call out to the browser with javascript to play the video fullscreen with the video element though
see https://github.com/godotengine/godot/issues/17495
make the attached vp8 and vp9 webm file plays properly <- this commit does not appear to create a significant performance regression, and I can confirm that the video which is broken in the reproduction plays correctly with up to 4x video streams being decoded concurrently. (probably need a better way to measure performance though than decoding 4 concurrent video streams)
perf.log i7-8750H perf-u620-i7-8650U.log
When playing a file that was encoded with vp9 and did not fail to playback in the previous version of godot the frame rate was roughly the same for both commits, and only slightly slower than the godot-videodecoder plugin...
i7-8750H
GODOT=./libvp9-thread-godot.x11.opt.tools.64
Godot Engine v3.3.5.rc.custom_build.63ac434b3 - https://godotengine.org
Playing char-waypoints-2022-01-27_20.08.38.webm 4x with builtin decoder
videos:0 fps:7673 ts:10091
videos:1 fps:6419 ts:15090
videos:2 fps:4412 ts:20090
videos:3 fps:4037 ts:25090
videos:4 fps:3663 ts:30090
videos:4 fps:3837 ts:35090
videos:4 fps:3711 ts:40090
videos:3 fps:4109 ts:45090
videos:2 fps:4364 ts:50090
videos:1 fps:4635 ts:55090
videos:0 fps:4994 ts:60090
GODOT=./libvp9-nothread-godot.x11.opt.tools.64
Godot Engine v3.5.beta.custom_build.1ede4ca4b - https://godotengine.org
Playing char-waypoints-2022-01-27_20.08.38.webm 4x with builtin decoder
videos:0 fps:1 ts:1
videos:0 fps:9063 ts:5111
videos:0 fps:8985 ts:10112
videos:1 fps:4349 ts:15112
videos:2 fps:6185 ts:20112
videos:3 fps:3474 ts:25112
videos:4 fps:2253 ts:30112
videos:4 fps:2986 ts:35112
videos:4 fps:4589 ts:40112
videos:3 fps:3449 ts:45112
videos:2 fps:3934 ts:50112
videos:1 fps:4388 ts:55112
videos:0 fps:4907 ts:60112
i7-8650U
GODOT=./libvp9-nothread-godot.x11.opt.tools.64
Godot Engine v3.5.beta.custom_build.1ede4ca4b - https://godotengine.org
Playing char-waypoints-2022-01-27_20.08.38.webm 4x with builtin decoder
videos:0 fps:2945 ts:6353
videos:0 fps:2898 ts:11353
videos:1 fps:950 ts:16354
videos:2 fps:492 ts:21355
videos:3 fps:333 ts:26354
videos:4 fps:232 ts:31360
videos:4 fps:238 ts:36357
videos:4 fps:234 ts:41359
videos:3 fps:274 ts:46359
videos:2 fps:304 ts:51355
videos:1 fps:336 ts:56358
videos:0 fps:366 ts:61359
GODOT=./libvp9-thread-godot.x11.opt.tools.64
Godot Engine v3.5.beta.custom_build.0c58454da - https://godotengine.org
videos:0 fps:2834 ts:6338
videos:0 fps:2554 ts:11339
videos:1 fps:954 ts:16340
videos:2 fps:515 ts:21339
videos:3 fps:366 ts:26342
videos:4 fps:279 ts:31342
videos:4 fps:278 ts:36341
videos:4 fps:281 ts:41340
videos:3 fps:299 ts:46342
videos:2 fps:320 ts:51342
videos:1 fps:341 ts:56345
videos:0 fps:363 ts:61344
Complete name : char-waypoints-2022-01-27_20.08.38.webm
Format : VP9
Width : 1 434 pixels
Height : 660 pixels
Complete name : vp8-encoded.webm
Format : VP8
Width : 1 920 pixels
Height : 1 080 pixels
Complete name : vp9-encoded.webm
Format : VP9
Width : 1 920 pixels
Height : 1 080 pixels
make the attached vp8 and vp9 webm file plays properly <- this commit does not appear to create a significant performance regression, and I can confirm that the video which is broken in the reproduction plays correctly with up to 4x video streams being decoded concurrently. (probably need a better way to measure performance though than decoding 4 concurrent video streams)
perf.log i7-8750H perf-u620-i7-8650U.log
When playing a file that was encoded with vp9 and did not fail to playback in the previous version of godot the frame rate was roughly the same for both commits, and only slightly slower than the godot-videodecoder plugin...
i7-8750H GODOT=./libvp9-thread-godot.x11.opt.tools.64 Godot Engine v3.3.5.rc.custom_build.63ac434b3 - https://godotengine.org Playing char-waypoints-2022-01-27_20.08.38.webm 4x with builtin decoder videos:0 fps:7673 ts:10091 videos:1 fps:6419 ts:15090 videos:2 fps:4412 ts:20090 videos:3 fps:4037 ts:25090 videos:4 fps:3663 ts:30090 videos:4 fps:3837 ts:35090 videos:4 fps:3711 ts:40090 videos:3 fps:4109 ts:45090 videos:2 fps:4364 ts:50090 videos:1 fps:4635 ts:55090 videos:0 fps:4994 ts:60090 GODOT=./libvp9-nothread-godot.x11.opt.tools.64 Godot Engine v3.5.beta.custom_build.1ede4ca4b - https://godotengine.org Playing char-waypoints-2022-01-27_20.08.38.webm 4x with builtin decoder videos:0 fps:1 ts:1 videos:0 fps:9063 ts:5111 videos:0 fps:8985 ts:10112 videos:1 fps:4349 ts:15112 videos:2 fps:6185 ts:20112 videos:3 fps:3474 ts:25112 videos:4 fps:2253 ts:30112 videos:4 fps:2986 ts:35112 videos:4 fps:4589 ts:40112 videos:3 fps:3449 ts:45112 videos:2 fps:3934 ts:50112 videos:1 fps:4388 ts:55112 videos:0 fps:4907 ts:60112i7-8650U GODOT=./libvp9-nothread-godot.x11.opt.tools.64 Godot Engine v3.5.beta.custom_build.1ede4ca4b - https://godotengine.org Playing char-waypoints-2022-01-27_20.08.38.webm 4x with builtin decoder videos:0 fps:2945 ts:6353 videos:0 fps:2898 ts:11353 videos:1 fps:950 ts:16354 videos:2 fps:492 ts:21355 videos:3 fps:333 ts:26354 videos:4 fps:232 ts:31360 videos:4 fps:238 ts:36357 videos:4 fps:234 ts:41359 videos:3 fps:274 ts:46359 videos:2 fps:304 ts:51355 videos:1 fps:336 ts:56358 videos:0 fps:366 ts:61359 GODOT=./libvp9-thread-godot.x11.opt.tools.64 Godot Engine v3.5.beta.custom_build.0c58454da - https://godotengine.org videos:0 fps:2834 ts:6338 videos:0 fps:2554 ts:11339 videos:1 fps:954 ts:16340 videos:2 fps:515 ts:21339 videos:3 fps:366 ts:26342 videos:4 fps:279 ts:31342 videos:4 fps:278 ts:36341 videos:4 fps:281 ts:41340 videos:3 fps:299 ts:46342 videos:2 fps:320 ts:51342 videos:1 fps:341 ts:56345 videos:0 fps:363 ts:61344Complete name : char-waypoints-2022-01-27_20.08.38.webm Format : VP9 Width : 1 434 pixels Height : 660 pixels Complete name : vp8-encoded.webm Format : VP8 Width : 1 920 pixels Height : 1 080 pixels Complete name : vp9-encoded.webm Format : VP9 Width : 1 920 pixels Height : 1 080 pixels
Thanks for your test, and hope this pr can satisfy some pepole's needs
@John-Gdi nice work！ Do you know how to fix VideoStreamWebm playing slowly in Html5? I really want to know where the problem is.
Video decoding will always perform poorly in HTML5 for two reasons:
- yasm assembly optimizations cannot be used.
- Decoding is performed on the CPU, via a pure C WebAssembly layer, which is slow.
There isn't a good way to work around this, as WebAssembly cannot make use of the browser's native video decoding functionality (which would allow for hardware-accelerated decoding). Theora might perform better as it's less demanding on the CPU, but I wouldn't expect it to be fantastic there either. @Calinou Can the technology in the link below make it possible to play webm videos smoothly? https://github.com/GoogleChromeLabs/webm-wasm
Can the technology in the link below make it possible to play webm videos smoothly? https://github.com/GoogleChromeLabs/webm-wasm
"This is a proof-of-concept and not a production-grade library."
It looks like it's for encoding video files from raw frames... Also probably not real time
@jamie-pate @Calinou Mainly I want to be able to play webm video smoothly in HTML5, but can't find any technology to replace it, although Ogg Theora video does what I want, it's too low definition. I'm wondering if it's possible to pre-decode webm-formatted videos while packing. . .
Ogg Theora shouldn't be low definition, it depends on how you encoded the video, like any encoder.
Is it because I set it wrong? The resolution of the video is not as good as the webm format.
@akien-mga
Your video encoding bitrate is set to 6 Mb/s, which is quite low by today's standards, especially for Ogg Theora which is less efficient than H.264 or VP9. For comparison, 1080p60 H.264 video on YouTube is around 10 Mb/s, but even that is considered low quality nowadays (and that's for a more efficient format).
To make things worse, you're using the Realtime encoding speed preset, which favors speed over quality (for a given bitrate). Use a encoding slower preset to get better quality with the same bitrate – it doesn't affect decoding speed.
Your video encoding bitrate is set to 6 Mb/s, which is quite low by today's standards, especially for Ogg Theora which is less efficient than H.264 or VP9. For comparison, 1080p60 H.264 video on YouTube is around 10 Mb/s, but even that is considered low quality nowadays (and that's for a more efficient format).
To make things worse, you're using the Realtime encoding speed preset, which favors speed over quality (for a given bitrate). Use a encoding slower preset to get better quality with the same bitrate – it doesn't affect decoding speed.
Great, after modifying the settings, the effect is much better now.
The current lack of seek() for the VideoPlayer node is what makes this node pretty much useless for me. From my tests there is no way to work around this issue.
Also all previous proposals and issues have been closed.
I'm very happy to see someone is working on this, may I ask what is left for it to be merged? Are there any plans for ogv seek support as well?
I'm very happy to see someone is working on this, may I ask what is left for it to be merged? Are there any plans for ogv seek support as well?
Please test the PR locally and report if it works :slightly_smiling_face:
Are there any plans for ogv seek support as well?
This was being worked on in https://github.com/godotengine/godot/pull/32539, but it needs to be salvaged.
@Calinou
Please test the PR locally and report if it works
I would love to, but the Windows artifacts are apparently already expired. And I don't feel competent enough to compile and build Godot with these changes.
I tested it on my end (GNU/Linux), and it doesn't appear to work. The video just goes back to the beginning. I'm using this file for testing: https://filesamples.com/samples/video/webm/sample_960x400_ocean_with_audio.webm
@John-Gdi Could you look into testing your PR with the file linked by YeldhamDev above?