ColossalAI
ColossalAI copied to clipboard
[FEATURE]: Basic Function of ColoTensors
Describe the feature
I found that ColoTensor lacks some basic functionalities.
- [x] initialized in shard mode from a torch tensor.
- [x] save and load in a distributed mode.
- [ ] support torch.einsum
- [ ] support conv2d
- [x] flexible assign process group for each tensor.
- [ ] inter-convert ColoTensor <-> torch.Tensor