Improvements on the `ffmpeg` port monitor to allow using different couples of coders/decodes

[S-Dafarra]

trafficstars

With this PR it is possible to select all the encoders and decoders available with ffmpeg. In order to do so, I had to allow specifying custom pixel formats, listing also all those available for each encoder/decoder. Moreover, I managed to fix a crash occurring when launching two port monitor instances.

I also added a utility to print the estimated bandwidth requested (it is a lower bound) and I made it work also in conjunction with yarp repeat.

We are currently using it with the NVIDIA hevc codecs in both Windows and Linux.

cc @mebbaid @GiulioRomualdi @traversaro

[S-Dafarra]

I can't ask for reviewers, so tagging people: @randaz81 @Nicogene @elandini84

[randaz81]

Thank you @S-Dafarra this is very useful!

[S-Dafarra]

Just to give an example, this PR can be used with a connection string as follows:

yarp connect /source /dest fast_tcp+send.portmonitor+file.image_compression_ffmpeg+recv.portmonitor+file.image_compression_ffmpeg+type.dll+custom_enc.hevc_nvenc+custom_dec.hevc_cuvid+pixel_format.23+cq.31+delay.2+tune.ull

where

• fast_tcp is the carrier used to stream the compressed image
• +send.portmonitor+file.image_compression_ffmpeg+recv.portmonitor+file.image_compression_ffmpeg+type.dll specifies to use the image_compression_ffmpeg port monitor on both the sender and receiver side
• +custom_enc.hevc_nvenc allows to use the hevc_nvenc encoder, that is a hevc encoder developed by NVIDIA working on GPU (you can check if the encoder is available by running ffmpeg -encoders). This compresses the image on the sender side.
• +custom_dec.hevc_cuvid uses the NVIDIA hevc decoder, i.e. the one that will uncompress the image on the receiver side. (check if available with ffmpeg -decoders)
• +pixel_format.23 specifies a pixel format that is compatible with both the encoder and the decoder (when in doubt, use any number and a message will be printed on both the sender and receiver side indicating the available pixel formats).
• +cq.31+delay.2+tune.ull specifies the compression level, and a low latency setting for the encoder. These are specific settings for the encoder. The available list of parameters for the encoder can be found with the command ffmpeg -h encoder=hevc_nvenc.

This command works if the source machine has the hevc_nvenc encoder available, and if the destination machine has the hevc_cuvid decoder available. If this is not the case, it is possible to use a yarp repeat to "bounce" on a machine that has the compatible encoder/decoder, or to choose a different couple of encoder/decoder according to the available ones (making sure that they both use the same algorithm, like h264, hevc, ecc.).

[S-Dafarra]

For the records, the hevc_nvenc encoder is not available in embedded boards like the xavier (at the moment at least). Back in time, I managed to have some good compression results and low latency using h264, with the following connection string

fast_tcp+send.portmonitor+file.image_compression_ffmpeg+recv.portmonitor+file.image_compression_ffmpeg+type.dll+custom_enc.libopenh264+custom_dec.h264+b.500K+allow_skip_frames.1+bufsize.10M+maxrate.1M+frame_rate.30

Here, by changing the b.500K and maxrate.1M parameters, we can specify the desired bandwidth (with a ~x2 multiplier). The first specifies the nominal bandwidth, the second the maximum. It is necessary the option allow_skip_frames.1 to enable these options. On the downside, it drops frames (everything becomes black) when it is not able to remain within the specified bandwidth.