ColossalAI icon indicating copy to clipboard operation
ColossalAI copied to clipboard
verified publisher icon hpcaitech

[autoparallel] refactor metainfo forward memory calculation

Open Cypher30 opened this issue 3 years ago • 0 comments

What's New?

In this PR, I refactor the forward memory calculation of former patched operations. After this PR, in SPMD solver, we will use the conservative way to estimate the memory, i.e. always believing that shape consistency will create a new tensor for TP.

Cypher30 avatar Dec 02 '22 03:12 Cypher30