[REQUEST] Enable both CPU and NVMe for optimizer

[shanhx2000]

Describe the solution you'd like Dear developers, I am trying to test the optimizer offloading function with deepspeed. I notice that when nvme offloading is enabled, all optimizers will go to NVMe, which misses the chances to put some in CPU. Although there is a configurable variable "ratio" in "offload_optimizer", it will result in some optimizers on GPU and some on CPU, leading an error when specifying "device" as "nvme".

An example expectation is: 0.05 optimizer states on GPU 0.55 optimizer states on CPU 0.4 optimizer states on NVMe