[BUG] The bug about the options of the Megatron-core, transformer-impl and flash-attention.

[Baibaifan]

Describe the bug Open --use-mcore-models and --use-flash-attn, set --transformer-impl local, and do not use flash-attention.

To Reproduce N/A

Expected behavior N/A

Stack trace/logs N/A

Environment (please complete the following information):

• Megatron-LM commit ID : ba773259dbe5735fbd91ca41e7f4ded60b335c52

Proposed fix N/A

Additional context N/A

[ethanhe42]

when you use --use-mcore-models,, you cannot use local. --use-flash-attn decides whether to use the OSS flash attention implmentation or cudnn implmementation.

[Baibaifan]

when you use --use-mcore-models,, you cannot use local. --use-flash-attn decides whether to use the OSS flash attention implmentation or cudnn implmementation.

hi @ethanhe42 ,I understand the process you mentioned, but currently there is a task warning in the configuration options, which is not very user-friendly.

[github-actions[bot]]

Marking as stale. No activity in 60 days.