element-hq
Changing the video codec from VP8 to H.264
Recent research has confirmed that the number of devices lacking H.264 decoding capabilities is negligible.
In contrast, many modern devices come equipped with native H.264 encoders and decoders. By switching to the H.264 codec, we can significantly reduce CPU usage. Especially mobile devices will benefit, where it can lead to better battery life.
Furthermore, we add a 540p simulcast video layer.
fixes https://github.com/element-hq/element-call/issues/3166 https://github.com/element-hq/element-call/issues/2579 https://github.com/element-hq/element-call/issues/2373
Current state
H264 video is still broken on iOS — other participants won’t receive video from an iOS device. This happens when the iOS device is the first to join a LiveKit room and starts streaming H264.
However, it's working if a non-iOS client is already streaming H264 before the iOS device joins. Note Using VP8 on the other client does not fix the issue — it must be H264.
What is the status on this one? Can the OP get a description what this is blocked by. (I do remember some device specific issues where some resolution just were not send at all?)
What is the status on this one? Can we OP get a description what this is blocked by. (I do remember some device specific issues where some resolution just were not send at all?)
H264 video is still broken on iOS — other participants won’t receive video from an iOS device. This happens when the iOS device is the first to join a LiveKit room and starts streaming H264.
However, it's working if a non-iOS client is already streaming H264 before the iOS device joins. Note Using VP8 on the other client does not fix the issue — it must be H264.
Test on safari on mac os works. (to make sure its ios and not safari in general
bump, any progress on this? what is blocking it? how about iOS 26? 👀
still blocked on Safari as h264 with e2ee as a first joiner with Safari does not work.
so this is not fixable until safari itself changes? which might never happen
Update
Things have changed after rebasing it on a large refactor including numerous livekit udpates. Those are the observations compared to previous ones:
H264 video is still broken on iOS — other participants won’t receive video from an iOS device. This happens when the iOS device is the first to join a LiveKit room and starts streaming H264.
This is not the case anymore. Its the opposite now: starting with iOS H264 (first to join the lk room) is realiably resulting in the other device receiving media. I do have slow network and matrix.org accounts (and matrix.org is also slow atm) so things might have started to work if i would have been a bit more patient
However, it's working if a non-iOS client is already streaming H264 before the iOS device joins. Note Using VP8 on the other client does not fix the issue — it must be H264.
If another client streams VP8 before it OCCASIONALLY does not work (not work = the vp8 client cannot see the ios stream, ios can see both. the local echo stream and the vp8 stream)
Both on vp8 seemed to always work. (independent on who joined first)
Tested on: Iphone + macOs (firefox)
@shootie22 @thorw4ld If you want to help out on testing this proceed as following:
on EW type in /devtools when sending the message a modal will appear with a custom EC url field.
on EX go to the settings tab and press the version number multiple times (i think 7) a devtool option will appear with a custom EC url filed
Use this URL: https://pr3345--element-call.netlify.app/room to test with the h264 build. (try all combinations of EW and EX using/not using it with all devices you have and report the observations here)
@toger5 thats awesome, thank you so much! I will try to test it out tonight after I get home and report back.
alright, I tested all combinations - result: as long as I'm using the EC version you provided on EX iOS, my phone will not transmit any video.
I tested using: My phone - iPhone 11 Pro Max on iOS 26.1, EX 25.11.3 Linux desktop - Fedora 43, Element Nightly 2025111401
the combination of EC version or which device joined first didn't seem to matter, as long as my phone was using the PR version of EC, it would send no video. :( I will note that my phone does indeed get hot. I don't think I noticed until now, but it gets hot on either version, stable or the one from this PR.
is there anything specific you'd like me to test or look for, perhaps in the developer menu?
Very nice initiative. But why h264 instead of h265? H264 will only be supported by safari, and maybe some implementations of webrtc that are not from Google. Chrome can't use it due to licensing concerns and will resort to software implementation (openh264) which is slower than vp8.
h265 on the other hand, has no licensing concerns as long hardware codec is used. A lot of modern chips (2018 and onwards) have both hardware encoding and decoding. Chrome supports it since the beginning of 2025, safari as well.
Using h265 with a fallback to vp8 would improve both quality and performance for many users as it's hardware accelerated and provides better compression.
Additionally it can unlock 1080p for many devices that were constrained by cpu before (its target bitrate is 3Mbps at 1080p, the same as vp8 at 720p).
Related: https://github.com/livekit/livekit/issues/2792
There are known compatibility issues. Worth investigating if they are resolved nowadays?
Yeah, I suggested it prematurely. It definitely has issues on Windows with Intel encoder. Issues listed in https://github.com/livekit/livekit/issues/2792 would not be blocking:
- Neither vp8 nor h264 support svc. It would be nice to have in h265, but even without it it's better in quality, compression and hardware support.
- Nvenc not supporting some resolutions. Not nice but element call is opinionated about supported resolutions anyway. Just need to keep that in mind, configuring it.
In my tests with element call. It works well between Mac/iOS/Andriod. Even at 1080p cpu usage is low. Windows has issues. It can decode h265 but produces broken encodings. I've spent some time on it, looks like a live kit bug. Linux I didn't test. Decoding would most like work. And encoding is likely challenging to enable.
I think it worth trying to at least allow to opt in for it on Mac/iOS/Android, and keep serving vp8 on Windows and linux for some time until the bugs are fixed.
Could a setting be added to let the user decide the codec? I think keeping a widely-supported codec the default is the right choice, but for certain use-cases something like h265 or av1 would be great.
