Infi_zc comments

Repositories
Issues
Comments

Results 5 comments of


                                            Infi_zc

Zero_Stage=1 results in higher TFLOPS?

@lhb8125 Do you understand the reason for this issue now?

[QUESTION] Why should CUDA_DEVICE_MAX_CONNECTIONS=1 should be set when using seq_parallel or async comm?

@wdykas Thanks for your reply! I have some further questions about that env var. 1. Does this environment variable have an upper limit? 2. What is the default value?

[Bug] backward NaN when using flash_attn_varlen_qkvpacked_func

@wangjunchao1118 hi, junchao, has this issue been resolved?

[ENHANCEMENT] Multi-token Prediction(MTP) support

> Hi guys, I'm glad to announce that the MTP in MCore has been merged into main branch. [dc385c7](https://github.com/NVIDIA/Megatron-LM/commit/dc385c76f3ced50f5b05597cbe09ab4ab5192b7d) > > Welcome to have a try and feel free to...

[ENHANCEMENT] Multi-token Prediction(MTP) support

> dc385c7 Thanks for the reply! The earlier worry was unnecessary, we can disregard this now.