Qt application with multiple AVPlayers stops working after running for a long period of time.

[noxee]

Hi,

First off thanks for writing such a great library. I'm currently experiencing an issue with a Qt application that creates multiple AVPlayers to display RTSP streams (the number of players can be anywhere between 1 - 9 players) and after running for a long period of time (any where from 2-3 hours) the AVPlayers stops updating the video (basically is freezes). Another oddity is that sometimes one of the AVPlayers will keep playing while all other players freeze.

I've tried looking through the logs but I'm not sure if it's something to do with ffmpeg or the way that I'm creating the AVPlayers.

I've tested it on my development computer as well as another machine I have.

The code uses the following versions of Qt, QtAV, and ffmpeg:

• Qt: 5.5.1 (compiled from source)
• QtAV: 1.9.0 (compiled from master)
• ffmpeg: N-77432-gc0f67e1 (compiled from source)

The code for creating the AVPlayer is as follows:

void CameraPlayer::createPlayer() {
    player = new AVPlayer(this);
    VideoOutput *vo = new VideoOutput(this);

    player->setBufferValue(30);

    QVariantHash va_opt;
    va_opt["display"] = "X11"; //"GLX", "X11", "DRM"
    va_opt["copyMode"] = "ZeroCopy"; // "ZeroCopy", "OptimizedCopy", "GenericCopy". Default is "ZeroCopy" if possible

    QVariantHash opt;
    opt["VAAPI"] = va_opt; //key is decoder name, case sensitive

    player->setOptionsForVideoCodec(opt);

    QVariantHash fmt_opt;

    fmt_opt["tune"] = "zerolatency,fastdecode";
    fmt_opt["video_size"] = "1280x720";
    fmt_opt["pixel_format"] = "yuv420p";
    fmt_opt["framerate"] = 30;

    player->setOptionsForFormat(fmt_opt);

    player->setRenderer(vo);

    QString url = QString("rtsp://%1:5544/%2_stream").arg(getNvrAddress(), cameraSlug);

    player->play(url);
    player->connect(player, SIGNAL(stopped()), this, SLOT(playerStopped()));
}

void CameraPlayer::playerStopped() {
    if (reconnectThread == NULL) {
        reconnectThread = new ReconnectThread(player);
        reconnectThread->start();
    } else {
        if (!reconnectThread->isRunning()) {
            reconnectThread->start();
        }
    }
}

The ReconnectThread is defined as follows (it's quite crude but it's basically seeing if the player has stopped and will attempt to start playing it again):

ReconnectThread::ReconnectThread(QtAV::AVPlayer *playerToCheck)
{
    player = playerToCheck;
}

void ReconnectThread::run()  {
    while(1) {
        QMutexLocker locker(&mutex);

        if (player->isPlaying()) break;

        player->play();

        sleep(10);
    }
}

When I was testing I tested the following scenarios:

• Development machine running one instance of the application with nine AVPlayers streaming (log file can be found here)
• Test machine running two instances of application with one screen running the application with six AVPlayers and a second screen running the application with four AVPlayers (log files can be found here and here respectively)

If you're able to point me in the right direction of where I can start troubleshooting this issue that would be greatly appreciated.

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/29910789-qt-application-with-multiple-avplayers-stops-working-after-running-for-a-long-period-of-time?utm_campaign=plugin&utm_content=tracker%2F307703&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F307703&utm_medium=issues&utm_source=github).

[wang-bin]

What about single avplayer

[noxee]

I've tried running the application with only displaying one camera and I encountered the same issue where it would stop updating the player after an extended period of time.

The logs from the application using a single camera can be found here.

Could it be an issue with FFmpeg becoming unresponsive due to the number of missed RTP packets?

I also changed the FFmpeg version I built QtAV with to 2.8.5.

[noxee]

Also one other thing. I've using ffplay to play the same stream as the one tested in the application and there doesn't appear to be an issue with the video not updating even after four hours. It's still reporting missed packets but it continues playing fine.

A sample of the output from ffplay is below.

[h264 @ 0x7ff0109a5cc0] left block unavailable for requested intra4x4 mode -1 at 0 58
[h264 @ 0x7ff0109a5cc0] error while decoding MB 0 58, bytestream 14222
[h264 @ 0x7ff0109a5cc0] concealing 1249 DC, 1249 AC, 1249 MV errors in I frame
[h264 @ 0x7ff010004ec0] RTP: missed 12 packets6KB sq=    0B f=9342/9342
[h264 @ 0x7ff01057eae0] error while decoding MB 112 64, bytestream -403
[h264 @ 0x7ff01057eae0] concealing 417 DC, 417 AC, 417 MV errors in I frame
[h264 @ 0x7ff010004ec0] RTP: missed 3 packets05KB sq=    0B f=9416/9416
[h264 @ 0x7ff010a2f340] left block unavailable for requested intra mode at 0 67
[h264 @ 0x7ff010a2f340] error while decoding MB 0 67, bytestream 4961

[wang-bin]

I see qtav drops many frames. I think frame drop is not necessary for this kind of streams. A frame drop option will be added this week.

[wang-bin]

Currently you can try connect(player, &AVPlayer::started, [=](){player->masterClock()->setClockType(AVClock::VideoClock);})

[noxee]

When I add that line of code (without changing any of the other settings) it introduces a "stutter" in the video. Basically the video pauses for a few milliseconds making it appear choppy (and slightly accelerated).

Could this be an issue with having the buffer value and/or frame rate set incorrectly?

[noxee]

@wang-bin Is there any update on this?

I'm still getting images that are stuttering. I've reduced the image quality and GOP on the cameras themselves and have set the BufferValue to 30 and removed the explicit framerate setting but I still occasionally see stuttering images (sometimes it's straight after application load, sometimes it's after the application has been running for a while)

Any other tips on what I can do to resolve this?

[khanhlv1407]

Hi! Is there any update on this issue? I got the same problem!

[khanhlv1407]

Finally, I put the timer to restart stream (stop/start) periodically. It can be solved temporary.

[fatemehkarimi]

I have the same issue even for single display. anyone found a solution? how can this problem be solved with timer?