Are there any examples of mixed input Gemm (A type fp16, B type fp8)?

[liligwu]

./tools/profiler/cutlass_profiler --m=16 --n=16 --k=1024 --A=fe5m2:\* --B=fe5m2:\*

works for me just fine, or any other combination of fp8 types, layouts etc. I also noticed that your A type if fp16 but B is fp8. Mixed input gemms are supported in CUTLASS but not plumbed into the profiler yet.

Originally posted by @thakkarV in https://github.com/NVIDIA/cutlass/discussions/1217#discussioncomment-8101728

54_hopper_fp8_warp_specialized_gemm (https://github.com/NVIDIA/cutlass/blob/main/examples/54_hopper_fp8_warp_specialized_gemm/54_hopper_fp8_warp_specialized_gemm.cu) is an example of inputs fp8, but I'm not able to find a mixed input (A type fp16, B type fp8). I also cannot find this setup of input under the condition that the accumulator is fp32 https://github.com/NVIDIA/cutlass/blob/9385141f19d87d9fb4c24fb29b65d0e8facd55a7/include/cute/arch/mma_sm90.hpp#L401C3-L402C51

Would anyone give an (A type fp16, B type fp8) example? Thank you!

[thakkarV]

https://github.com/NVIDIA/cutlass/tree/main/examples/55_hopper_mixed_dtype_gemm

[mnicely]

@liligwu has you resolved your issue?

[liligwu]

@liligwu has you resolved your issue?

Yes, thanks @thakkarV's help.