[FEATURE] launch physical meshes after compilation

[ZYHowell]

This can be a starting point to learn runtime_emitter.

Background

In Pipeshard Parallel, the final compilation step is to interpret the solution into a configuration containing all information about computation, communication, parameter, output and debug information. Now the instruction_lists and executable_configs are indexed by MeshHostWorker, which means we have to give the emitter a physical mesh group(see here). The final target is to run all steps in the compilation(except for profiling) without using GPU resources.

TODO

• [ ] call SymbolicReshardingTask.put_tasks in PipeshardMeshDriverExecutable.__init__. The same applies for the SymbolicBroadcastReshardingTask;
• [ ] replace the mesh_group by sliced_virtual_meshes and move the two lines out of the function. Maybe we can create a VirtualMeshGroup to provide interfaces. Some operations will be skipped;
• [ ] add skipped operations from last step in PipeshardMeshDriverExecutable.__init__.

[haifaksh]

working on it