Reza Yazdani

icon company @microsoft icon location Vancouver, Canada icon location Machine Learning Engineer

Results 21 issues of Reza Yazdani

Small fix in injection module to replace transformer

Fixing several issue in API and kernels to run inference with model-parallelism

Triangular mask support for Transformer kernel

1 comment

Adding the new version of SoftMax for Transformer kernel to support Triangular mask used in GPT-based models. This addresses https://github.com/microsoft/DeepSpeed/issues/828. TODO: Add a unit test for guarding against this type...

CPU-Adam: add compile-flag to enable param-copy from CPU to GPU

1 comment

This addresses https://github.com/microsoft/DeepSpeed/issues/1279

Fix the file-paths for the auto-generated hip files through JIT

1 comment

Transformer/fix layer norm

1 comment

This PR addresses https://github.com/microsoft/DeepSpeed/issues/581

Refine quantizer for supporting larger hidden-dim and group size

1 comment

Add inference comm

2 comment

Add shared libs

1 comment

Modify unit test to cover more cases

1 comment

This PR adds two set of modification to the unit tests for better test coverage on optimizer functionality: 1. Increase the parameters groups to catch errors better at optimizer step...