core: frontend: video streams -> multimedia pipeline manager

[ES-Alexander]

Current behaviour

The Video Streams interface page is currently device-focused, which

1. can involve a lot of scrolling through devices to find active streams,
2. is poorly scalable to handling desirable extensions to the pipeline systems (like audio, external processing, and (re)encoding), and
3. is poorly equipped to display even what the current pipelines involve (which can be important for debugging and understanding resource usage, e.g. if one video source has several WebRTC outputs active).

Expected or desired behaviour

It seems more intuitive (and scalable) to change the page to being pipeline-focused, where the user can see the details of their active pipelines (likely with a simplified interface in standard mode and a detailed one in pirate mode), and can more directly control the pipeline processing and outputs (including disabling or removing select parts they don't want to keep).

Through discussion with @joaoantoniocardoso, we've come up with some interface ideas that should hopefully cover this, and allow BlueOS to advance into a maintainable future of multimedia handling:

• [ ] "Known Inputs" list/gallery at the top
  • [ ] simple icons for input type (e.g. camera / microphone) and connection type (e.g. USB / network)
  • [ ] input configuration (e.g. for camera settings like brightness and exposure, which affect all streams from that camera)
    • [ ] should support ONVIF and UVC/V4L2 style configuration as relevant
    • [ ] ideally would be extensible (e.g. if users have some other camera driver, or want to support a different ONVIF flavour)
      • as a workaround we could require custom drivers to provide a V4L2-style API for configuration, but having custom configuration support through plugins would be preferable
  • [ ] status icons/counters for the configured pipeline(s)
    • green for all active and working
    • yellow for some errors
    • red for all errors
    • grey for pipeline configured but disabled
  • [ ] button to register new inputs (e.g. undiscovered IP cameras)
• [ ] Pipelines underneath
  • [ ] three-section structure, with inputs, processing, then outputs
    • [ ] displayed visually, with process-style nodes and arrows
    • [ ] followed via wizard-style pipeline configuration popup
  • [ ] longer term the "processing" section should be extensible (e.g. with user-provided functions/programs/elements)
  • [ ] all configured outputs are displayed
    • [ ] outputs can be added, configured, disabled, or deleted as relevant
    • [ ] issues with an output should be indicated on that output, and the arrow(s) leading to it
      • [ ] red arrows should provide access to some relevant debugging information (e.g. internal pipeline structure, specific failing gstreamer element, gstreamer pipeline logs, etc)

Prerequisites

• [X] I have checked to make sure that a similar request has not already been filed or fixed.

[ES-Alexander]

Where relevant, we could also consider adding resource usage indicators (e.g. to show CPU and memory usage of elements, and potentially also to show network bandwidth consumption of outputs).

[patrickelectric]

To add on this, the proposal appears to be direct related to the software architecture proposed here: https://github.com/mavlink/mavlink-camera-manager/issues/274#issuecomment-1692115707

[joaoantoniocardoso]

Where relevant, we could also consider adding resource usage indicators (e.g. to show CPU and memory usage of elements, and potentially also to show network bandwidth consumption of outputs).

This would be very useful for development/debugging. Here is a related plugin called gst-instruments .

[ES-Alexander]

This could also be used for modification of the standard gstreamer pipeline, which would be useful for cases like this one.