ColossalAI icon indicating copy to clipboard operation
ColossalAI copied to clipboard

Published 20 hours ago • verified publisher icon hpcaitech

[BUG]: the autoparallel compatibility test uses same input data for different data parallel rank

Open YuliangLiu0306 opened this issue 2 years ago • 0 comments

🐛 Describe the bug

The compatibility test for combination auto-parallel with Gemini used same input data for different data parallel rank. Therefore, we can't check whether the gradient hooks have been activated correctly or not.

Environment

No response

YuliangLiu0306 avatar Feb 16 '23 08:02 YuliangLiu0306