ColossalAI icon indicating copy to clipboard operation
ColossalAI copied to clipboard

[autoparallel] add data_ptr

Open super-dainiu opened this issue 2 years ago • 1 comments

What's new?

Since all data in OperationData are meta tensors, their tensor.data_ptr()=0. I replace the original tensor.data_ptr() with a customized function generated by uuid4().

Why data_ptr?

Generally speaking, for fine-grained memory calculations, I need to specify which tensors are the same tensors which occupy the same piece of memory on the devices.

Test

image

super-dainiu avatar Nov 08 '22 10:11 super-dainiu

This is still a draft?

FrankLeeeee avatar Nov 11 '22 01:11 FrankLeeeee