Aleksandar Samardžić

This is not yet ready for merge, there are number of tests failing. I hope to revisit it soon.

This one is finally ready for merging.

> @alexsamardzic any idea why CI tests fail? It looks like `to_sparse` aten implementation is never called... Typo :flushed:

@pytorchbot merge

This PR is intended to extend existing support for `S8`/`U8` dtypes for mixed GEMM with the same kind of support for `S4`/`U4`. PR is opened for an initial review, there...

> @alexsamardzic , thanks for the PR. I am going to try 1. can you give write privilege to your branch? Done. (FWIW, my time zone is GMT+1 - sorry...

Thanks, let me try to make an update along the line of approach 2.

A branch with an implementation according to the approach 2 above is [here](https://github.com/alexsamardzic/cutlass/tree/4bit-support-mixed-dtypes-gemm-2). It has some quirks too: 1. The code at the threadblock level, calling `mma.transform()` and `mma()` that...

Any feedback on my last comment above?

Hi @manishucsd, thanks for the additional clarifications - I think by far I understand well the various components of changes needed for `S4`, but it was still helpful to make...