Megatron-LM icon indicating copy to clipboard operation
Megatron-LM copied to clipboard
Published 1 month ago • verified publisher icon NVIDIA

Enabling UCC backend for PP communication

Open youngeunkwon0405 opened this issue 1 year ago • 0 comments

This MR provides an interface to enable the UCC backend for PP communication. To enable the UCC backend, set the following argument:

training.model.pipeline_model_parallel_comm_backend=ucc

Requires a related NeMo PR (link).

youngeunkwon0405 avatar Sep 24 '24 21:09 youngeunkwon0405