int4 inverse quantization and gemm on existing templates

[kahakuka]

Is it difficult for me to achieve the fusion of int4 inverse quantization and gem using the existing template? What suggestions do you have?

[zjing14]

HIP does not support sub-byte data types. Are you using int4x2?

[kahakuka]

@zjing14 Thank you for your answer.Not use int4x2.It quantifies fp16 into int4 according to a certain pattern and stores it in a uint32 type.The paper introduces it this way.

[kahakuka]

@zjing14 Is it easy to implement the integration of int4+gemm on composable_kernel by referring to the method of mma in llm-awq?

llm-awq：The processing of int4 dequation can refer to this file. https://github.com/mit-han-lab/llm-awq/blob/main/awq/kernels/csrc/quantization_new/dequantize.cuh

[huanrwan-amd]

@xiabo123 Can you provide more context about your request? what do you intend to do for your project? If you have code snippet to share that would be helpful.