Performance Issues with Portrait Images

[Skylion007]

So I've managed to get this code optimized to run at an acceptable framerate on mobile. However, I've noticed one interesting particularity. When holding the phone in portrait mode (creating a video with a different aspect ratio than the underlying video frame), I noticed that there is a large drop in framerate when holding the phone in portrait mode. This creates as much as a 25% drop in performance. Are there any parameters that may explain the difference in performance?

[MariasStory]

Hi @Skylion007,

WebGazer.js is a cool idea. I'll try it in my project ;-)

I think that the performance drop that you see may come from the frame size. On mobile, the frame dimensions change when you rotate a phone and the frames are bigger in portrait mode.

Greetings, Tolik

[Skylion007]

@MariasStory They aren't necessarily bigger, but of different dimensions. However, apparently Chrome now has support for square resolutions from the Webcam via user media constraints according to this blog post. Note how bizarely only constaints higher than 289 work for the square aspect ratio. Also if not configured properly, Chrome will apparently stretch and deform the video so that could be a contributing factor. Another issue I have just read about is that when Chrome rotates the video, it does so using the CPU!!! That is a big no no on mobile, but it looks like no one has submitted a patch to Chromium yet. The final issue I have found it is that Chrome for Android actually lowers the framerate in WebRTC when under high CPU usage. This may be able to be bypassed using the min and max framerate in Chrome's constraints. Let me know if you discover any improvements and feel free to commit them.