ColossalAI
ColossalAI copied to clipboard
[autoparallel] add binary elementwise metainfo for auto parallel
What's New?
In this PR, I add binary elementwise metainfo for auto parallel. I also fix some annotations.
NOTE: Currently this metainfo is only a rough estimation for binary elementwise operations, in some cases, memory estimation could not match with the real one. Moreover, the memory footprint of different operations could be different, I will dive into those problems later.