Jesse Cai

@pytorchbot rebase

@pytorchbot rebase -b viable/strict

@jcaip has imported this pull request. If you are a Meta employee, you can view this diff [on Phabricator](https://www.internalfb.com/diff/D42199001).

@jcaip has imported this pull request. If you are a Meta employee, you can view this diff [on Phabricator](https://www.internalfb.com/diff/D42199001).

@jcaip has imported this pull request. If you are a Meta employee, you can view this diff [on Phabricator](https://www.internalfb.com/diff/D42199001).

@pytorchbot merge

@pytorchbot rebase

@jcaip has imported this pull request. If you are a Meta employee, you can view this diff [on Phabricator](https://www.internalfb.com/diff/D42199001).

I believe support for windows support for cusparselt was added [here](https://github.com/pytorch/pytorch/commit/2028987bf72739ac5b2bd342fd36da3442cc3ca0). However have not tried this personally. CUTLASS should be supported. Also FYI I plan to add better cusparselt detection...

@cpuhrsch @alexsamardzic Could we also run a test for m, n, k = (3072, 768, 3072) as well? I think the second linear inside the ffn will do a matmul...