DeepSpeed
DeepSpeed copied to clipboard
pre-commit check for torch.cuda in code
This PR add checks for CUDA specific code in DeepSpeed. The purpose is to avoid accidental usage of CUDA code in new code.
Two kinds of code are checked against:
torch.cudacode which calls PyTorch cuda runtime.tensor.cuda()which converts tensor to cuda device.
Also port remaining cuda specific code which can be ported.