rework video streaming code

[jaxxzer]

The video receiver code is a mess of spaghetti, and video streaming has suffered a long running blight of bugs and spotty support. It needs some tender love and care.

I think that we should define exactly what we want to achieve, make a plan and execute it. @tcanabrava will be available to help us with the api design and execution.

I propose this list as a general starting point to aim for in terms of functionality. Some of these features are already (dubiously) implemented:

• Support formats: H264/265, MJPEG, YUYV (raw)
• Support RTSP and UDP
• Support any resolution
• Support at least 4 concurrent streams, display and recording
• Solid support for video streaming on all of our platforms

This is open for discussion and input. I'd like to establish what the community needs and what will be accepted for merge. We have some preliminary UML work to demonstrate how this could be accomplished, and we will update this as it develops.

[DonLakeFlyer]

"and crashes"

@dogmaphobic Needs to be directly involved in this

[MatejFranceskin]

I am also interested in this. Would it be possible that you share perliminary UML work?

[tcanabrava]

There’s an open wip PR that you can check

On Tue, 12 Nov 2019 at 10:03 Matej Frančeškin [email protected] wrote:

I am also interested in this. Would it be possible that you share perliminary UML work?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mavlink/qgroundcontrol/issues/7842?email_source=notifications&email_token=AA6XGUFA3BKALSDRXG6YIODQTJWN5A5CNFSM4I4DMVW2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDZQ3FY#issuecomment-552799639, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA6XGUB7GCZFO5WPCX4EMA3QTJWN5ANCNFSM4I4DMVWQ .

[jaxxzer]

The wip pull request is here.

@tcanabrava can you post your UML here?

[jaxxzer]

ping @tcanabrava

[tcanabrava]

Right now I have two branches, one with a simplified camera handling (handles one camera) and another with a complex camera handling (handles 40+ cameras).

The simplified camera handling has the following features:

• play video
• pause video

Rationale:

• The old QtGstreamer code is removed in favor of the new QmlGlSink, this alone removes around 6k lines of code, simplifying things.
• The new QmlGlSink works in OpenGL so we have a much nicer CPU use as there's no need to copy bitmap buffers all around
• The pipeline is created via gst_parse, significantly simplifying pipeline code (and if we know the pipeline we can add only the plugins that we use for the binary, no need to add the full gstreamer)
• Play / Stop happens in OpenGL render thread, this means that we can't call ->play() ->stop() directly as we used to do, communication should be done by signals / slots (that also helps cleaning out code and detangling - for instance, in master vehicle calls camera (but shouldn't)).

The Multiple Camera approach is build on top of the single camera approach, but adds a few other things:

• A model that controls the amount of cameras that are being currently displayed
• Settings per camera
• Control of the video per video item.
• (tested in a i5 laptop and we could easily handle 40 streams)

[tcanabrava]

[tcanabrava]

This draft uml does not represents correctly the current implementation as the current implementation is much more complex, but it gives us a more understandable approach of the current architecture of video in qgc.