[REQUEST] Is the cuda unified memory technology used in deepspeed? Where is its API?

[gukejun1]

such as that title, Is the cuda unified memory technology used in deepspeed? Where is its API?

[tjruwase]

No, cuda unified memory is not used in deepspeed.

[gukejun1]

@tjruwase Thank you for solving the puzzle. There are two questions:

1. What are the heterogeneous memory optimizations performed by DeepSpeed? Which of the following APIs is used? 2, Does it provides the GPU memory swapping mechanism with the CPU main memory ? What is this mechanism like? What about APIs?

[tjruwase]

The heterogenous memory optimizations are

1. ZeRO-Offload
2. ZeRO-Infinity.
3. ZeRO-Inference