Yonghao Zhuang

icon indicating a website link zyhowell.github.io

icon company Carnegie Mellon University icon location Pittsburgh, PA icon location CMU CSD PhD student | distributed sys/MLsys | previous RA@ucbrise, undergrad@SJTU-ACM-Class-2018

Results 41 comments of Yonghao Zhuang

Multi-GPU Context Parallel Mamba2

It looks like you are using the `process_group` for context parallel, while it was designed for the tensor and sequence parallel. As long as TP and CP are compatible, I...