Improved Resource Constraints for Transcoding

[ericxtang]

We currently use concurrent sessions to limit the amount of resources used on a GPU, to make sure we don't overwhelm the GPU. This is a coarse approximation - and it's influenced by the input/output video size, complexity of the video.

Can we identify another metric that more closely resembles the utilization of GPU encoders/decoders?

[leszko]

@thomshutt Is it something you want to take into your teams' pipeline?

[yondonfu]

An additional note - this may be further complicated if an additional capability is requested alongside transcoding i.e. running ML model inference on the decoded frames of the video.

[thomshutt]

@leszko I've added it for consideration as part of Q3 work, since it will need some high-level thought on the approach we take as well as (as @yondonfu mentioned) tying in with how we account for / compensate any non-transcode tasks

[cyberj0g]

Related: #2446