ColossalAI icon indicating copy to clipboard operation
ColossalAI copied to clipboard

[FEATURE]: Refactor CommSpec

Open YuliangLiu0306 opened this issue 2 years ago • 0 comments

Describe the feature

The previous CommSpec used in Auto-Parallel module contains some attributes which is not necessary to describe a communication operation, such as ShardingSpec, DeviceMesh.

Previously,It has two main functions:

  1. Compute the communication cost which will be used in auto parallel solver.
  2. Convert the communication spec to real action which will be used in runtime.

However,the first function may not be necessary if we jump out of the auto-parallel scenario.

To make a clean design, the new CommSpec will just contains the attributes to decribe a communication operation and supply a function to apply the CommSpec to real execution.

image

YuliangLiu0306 avatar Mar 07 '23 09:03 YuliangLiu0306