Baibaifan comments

Repositories
Issues
Comments

Results 1 comments of


                                            Baibaifan

[BUG] The bug about the options of the Megatron-core, transformer-impl and flash-attention.

> when you use --use-mcore-models,, you cannot use local. --use-flash-attn decides whether to use the OSS flash attention implmentation or cudnn implmementation. hi @ethanhe42 ,I understand the process you mentioned,...