transformers icon indicating copy to clipboard operation
transformers copied to clipboard

[WIP] Quartet QAT support

Open BlackSamorez opened this issue 4 months ago • 1 comments

This PR adds support for the Quartet QAT method.

The goal of this PR is to integrate inference and training support for the Quartet QAT method. That would allow to perform both forward and backward passes in MXFP4, allowing for very fast training on Blackwell GPUs.

Currently, we're working on the kernels here, here and here (some of the libs aren't public yet). We're planning to release the first version of the kernels this week and have optimized performance by end of June.

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag members/contributors who may be interested in your PR.

BlackSamorez avatar Jun 09 '25 12:06 BlackSamorez