High VRAM usage

[LeroyR]

Issue Summary

Our setup with Openpose 1.5.1 uses <2GB VRAM (cuda 10.1.105, cudnn 7.6)

Openpose(master) consumes 4.4GB for the same task, same models (cuda 11.6, cudnn 8.4.1)

• CUDA version 11.6
• cuDNN version: 8.4.1
• GPU model 1060 6GB

We have some 1060 3GB systems where this blocks upgrades.

see #2008 #1946