MS-AMP icon indicating copy to clipboard operation
MS-AMP copied to clipboard

Published 20 hours ago • verified publisher icon Azure

V0.4 Release Plan

Open cp5555 opened this issue 1 year ago • 1 comments

Release Manager

@cp5555

Endgame

  • [x] Code freeze: Feb. 9th, 2024
  • [x] Bug Bash date: Feb. 12th, 2024
  • [x] Release date: Feb. 23rd, 2024

Main Features

MS-AMP O3 Optimization

  • [ ] 1. Support auto scaling factor tuning (ASFT) for FP8 collective communication (Related to #41, #140)
  • [x] 2. Support PyTorch FSDP (Related to #122)

MS-AMP Improvement

  • [ ] 1. Move extension installation from post install to setup.py (Related to #43)
  • [x] 2. Improve FP8 kernel performance in MS-AMP (#132)
  • [ ] 3. MS-AMP support on different devices (Nvidia A100 and AMD MI300X)

MS-AMP Examples

  • [ ] 1. Release the datapoints (Related to #115)

Backlog

FP8 on Activation

cp5555 avatar Nov 06 '23 00:11 cp5555

Will move all the non-started work items to next release v0.5.

tocean avatar Feb 29 '24 02:02 tocean