[Draft] Throttling poc for CUDA staging buffers allocation

[amastbaum]

What?

Describe what this PR is doing.

Why?

Justification for the PR. If there is an existing issue/bug, please reference it. For bug fixes, the 'Why?' and 'What?' can be merged into a single item.

How?

It is optional, but for complex PRs, please provide information about the design, architecture, approach, etc.