DeepSpeed icon indicating copy to clipboard operation
DeepSpeed copied to clipboard

Published 20 hours ago • verified publisher icon microsoft

Add FALCON Auto-TP Support

Open RezaYazdaniAminabadi opened this issue 1 year ago • 1 comments

This PR adds the support for running Falcon-40B on multiple A100 GPUs. To run the test for this model you can use this PR on DeepSpeedExample repo.

RezaYazdaniAminabadi avatar May 30 '23 22:05 RezaYazdaniAminabadi

@RezaYazdaniAminabadi could we merge this?

lanking520 avatar Jun 08 '23 16:06 lanking520