ColossalAI icon indicating copy to clipboard operation
ColossalAI copied to clipboard

[FEATURE]: Basic Function of ColoTensors

Open feifeibear opened this issue 3 years ago • 0 comments

Describe the feature

I found that ColoTensor lacks some basic functionalities.

  • [x] initialized in shard mode from a torch tensor.
  • [x] save and load in a distributed mode.
  • [ ] support torch.einsum
  • [ ] support conv2d
  • [x] flexible assign process group for each tensor.
  • [ ] inter-convert ColoTensor <-> torch.Tensor

feifeibear avatar Jun 21 '22 08:06 feifeibear