Pulldown algorithm when playing 50 fps content on 60 hz

[karathb]

Hi all! We were recently upgrading the ExoPlayer in a project and we experienced some weird behavior. Earlier we were using 2.10.8, and after the upgrade we have 2.12.1. I know it is still not the newest version, but in the business world these things take time, so for the sake of the example, let's just not focus on this part of the issue for now. (I would be more interested in understanding the reason behind the issue).

As mentioned in the title, we have a 50 fps live stream. We play it on a custom Android Tv device (dedicated hardware, android version and firmware, every test using the exact same setup). The display is on 60 hz. With 2.10.8, it looked smooth, visually there was no stuttering, at least not something that the human eye would notice. With 2.12.1 however, sometimes there is a black frame displaying. Only one frame. The problem is not consistent, not happenning on every display (tv, monitor). So far we were not able to tell what kind of displays have this issue. We start playing the stream and after a while, the stream starts blinking. Blinking means that black frames are popping up at random. The beginning of the problem can be delayed if we start playing around with some LoadControl parameter. If the buffer is bigger, the problem appears later, but after ~30 minutes, it is happening anyways and it is pretty frequent from that point on. Like there is a black frame in every ~30 seconds.

There are no exceptions like BehindLiveWindowException and also the frames are not dropped. Based on the logs, nothing bad is happening, and we also did not find anything worrying using Charles proxy. We used the same stream with the older player version, and it was working just fine on the problematic displays.

Also if we switch the display to 50 Hz, the problem is solved, stream is playing smoothly.

Does anyone have a guess what can couse this issue? Did the pulldown algorithm change? Is there a way to solve this problem? Can we expect the latest version of the player to have improvements in this area? (yeah, I know, I should just test it, but it is not that easy because of some circumstances, but we are on it :) )

I can probably share some samples if needed, but first I would be just interested in some theoretical discussion, maybe somebody already knows the reason for this.

Hope I was clear enough, and thanks in advance!

Edit: here is a video demonstrating the problem: https://drive.google.com/file/d/1Nqf4GKQy84oh8h5_MbRtygZlRo8rmYhG/view?usp=sharing

[tonihei]

@christosts Do you have an idea what could be causing the black frames?

The beginning of the problem can be delayed if we start playing around with some LoadControl parameter. If the buffer is bigger, the problem appears later, but after ~30 minutes, it is happening anyways and it is pretty frequent from that point on. Like there is a black frame in every ~30 seconds.

This is an interesting statement because I wouldn't have thought the load control can have an influence. Can you check what the buffer levels (player.getTotalBufferedDurationUs()) are during this time? And whether the player switching from loading to not loading has any influence (listen to EventListener.onIsLoadingChanged)?

With 2.10.8, it looked smooth, visually there was no stuttering, at least not something that the human eye would notice. With 2.12.1 however, sometimes there is a black frame displaying.

I can't spot anything between these two versions, but there were some changes to the frame release logic in December 2020 that are in 2.13.0 only. There is no particular reason they make a difference to your problem though.

[karathb]

This is an interesting statement because I wouldn't have thought the load control can have an influence. Can you check what the buffer levels (player.getTotalBufferedDurationUs()) are during this time? And whether the player switching from loading to not loading has any influence (listen to EventListener.onIsLoadingChanged)?

loading does not have any influence, that value is not changing during this event. Sometimes it is loading when the flicker is happening, but mostly it is not loading

the buffer is between 2000-3500

One more info, so far it seems like that this is only happening on FullHD displays/resolution. On 4k devices, it is not happening

[karathb]

Hmmm... we found something interesting. It seems like that not the stream is blinking (wow...), but rather the whole screen. If we put UI elements on the top of the player, those are also blinking (e.g.: another activity, or anything basically). It is crazy.

So if the player is playing a 50 fps content on a 60 Hz FullHd display (AndroidTv), then the whole screen is flickering.

There is no screen flickering in the following cases:

1. with the mentioned older player version
2. if the player is not playing anything
3. if the display is on 50 Hz
4. on 4k displays

Can the player maybe interfere with the AndroidTv's framerate-conversion logic? I don't really understand unfortunately the connection between a physical display + AndroidTv + ExoPlayer.

[karathb]

Did you have a chance to have a look at it @christosts ? Any idea? Thanks in advance!

[christosts]

So if the player is playing a 50 fps content on a 60 Hz FullHd display (AndroidTv), then the whole screen is flickering.

If that's the case, I don't think this can be related to LoadControl or the pull-down algorithm. There have been some changes in the pull-down algorithm, but those were added in 2.13, including a call to set the surface frame rate which is available on Android S onwards.

@karathb Is it possible to use the ExoPlayer demo app on r2.13.2 on the specific device with your 50fps content to see how it works?

I'll take a look if 2.12.1 added a call to a platform API that wasn't there before.

[karathb]

Thanks for your feedback! I will see if I can do a test with the demo app or not, but I am not too optimistic about this. I will get back to you.

Just let me know if you could check if there is any nasty platform API usage that can result in this.