A lowLatencyVideo player implementation

[swimfish09]

Now when set player usage to lowLatencyVideo, there will be no audio and the video can not play smoothly if it has a moving object. Maybe it is better to set speed to 1.1 or 1.2 to reduce latency. Just like below code. ijkplayer low latency implemention

static void control_max_delay_duration(FFPlayer *ffp, int max_delay_ms, int network_jitter_ms, float new_play_rate) 
{
    if (max_delay_ms == 0)
		return;
	VideoState *is = (VideoState*)handle;
	uint32_t cached_duration = get_video_queue_cached_duration(is);
	if (cached_duration > max_delay_ms + network_jitter_ms && ffp->pf_playback_rate == 1.0)
	{
		//set speed to 1.1 or 1.2
		ffp_set_playback_rate(ffp, new_play_rate) ; 
	}
	else if(cached_duration <= max_delay_ms && ffp->pf_playback_rate != 1.0) 
    {
               // reset
		ffp_set_playback_rate(ffp, 1.0);
	}else{
        ;// do nothing...
    }
} 

[SuRGeoNix]

@swimfish09 Please don't create duplicate issues. This one is the same with #156

You are right, low latency can be improved and allow audio as well. I will review this on the next updates.

By the way, did you try already to increase speed to normal player (eg to 1.1) and see if that looks good for you?

You could increase Config.Player.LowLatencyMaxVideoFrames and Config.Player.LowLatencyMaxVideoPackets of the current implementation to have more smooth motion.

[swimfish09]

Hi @SuRGeoNix I'm Sorry for create duplicate issues. I have already tried to increase speed to normal player (eg to 1.1) and it is not very good at soon but after some time the latency reduced and looks great. Maybe i should seek to the latest timestamp after the stream being opened. the code as belows:

TimeSpan timeSpan = new TimeSpan(player.BufferedDuration);
Console.WriteLine($"Player: {Player.PlayerId},BufferedDuration: {timeSpan.TotalMilliseconds}");
if(timeSpan.TotalMilliseconds > 100 && Player.Speed == 1.0f)
{
    Player.Speed = 1.2f;
} else
{
    Player.Speed = 1.0f;
}

[SuRGeoNix]

Initially it will wait to fill the buffer and frames. You can try to change the configuration (Config.Player.MinBufferDuration)

[SuRGeoNix]

I have added a new low latency screamer mainly for cameras which supports audio as well. It will not drop any frames for now but it will not keep the frame rate either to ensure zero latency. I will review this, let me know your thoughts, I might just increase the speed. Could cause some high GPU as it will render too fast sometimes.

https://github.com/SuRGeoNix/Flyleaf/commit/d8737f2beece5bb853697833bfe4aec5f70ac987

[swimfish09]

Hi @SuRGeoNix Usage.LowLatencyAudioVideo can keep low latency, but it can not play smoothly.

[SuRGeoNix]

That's the cost of zero-latency, you can't have both. I could review to have a variable how much of "zero-latency" you want so you could play smoother. I was thinking something like Max Buffered Duration (e.g. 5 seconds) so it will play normal and when it passes the MBD it will increase (speed will be relative to MBD, e.g. the more distance from "now" the playback will be faster) the speed or just jump directly to the "zero-latency". Let me know your thoughts and I will try to implement this next week.

[swimfish09]

Latency less than 500ms is better. Just increase the speed, could cause high GPU or CPU usage particular with multiplayers.

[SuRGeoNix]

Can you provide (email) me a test camera (with audio) so I can implement it better in real conditions?

[SuRGeoNix]

I've made some changes. Trying to merge the normal playback with the low latency so you will not need to set different usage for the player. You can try it and let me know how it goes. Note by changing the latency will affect also the min/max buffered duration configuration:-

Config.Player.MaxLatency = 1500 * 10000 // For 1.5 second latency

[SuRGeoNix]

I'm closing this and feel free to re-open after testing it and still having issues