apache
[BugFix][Codegen, CUDA] Fix faulty codegen for FP8
Fixed bug where CUDA codegen produces faulty code when a vectorizable BufferLoadNode contains a Float8 type.
Codegen generated the invalid signature
make___nv_fp8x2_e5m2(param_0[v_.x], param_0[v_.y]) where "param_0" is of type __nv_fp8_e5m2* __restrict__.
This commit adds a missing check is_float8() for CodeGenCUDA::PrintVecElemLoadExpr that is called for vectorizable BufferLoadNodes. Which instead correctly generates the signature _nv_fp8x2_e5m2(make_float2(static_cast<float>(param_0[v_.x], static_cast<float>(param_0[v_.y])))
Additionally this commit removes the added "make_" prefix for float8 in CodeGenCuda::PrintVecConstructor as the correct way to instansiate an nv_fp8x2_[e5m2/e4m3] is through the _nv_fp8x2_[e5m2/e4m3] constructor itself.
thanks @AntonMoberg , @MasterJH5574 would be great if we can validate this PR
Thank you @AntonMoberg! Would you mind providing an example which can reproduce the error?
Hi @tqchen & @MasterJH5574! I am trying to produce a minimal reproducible example but it is proving a bit challenging as the error only occurs in some specific scenarios. However, during this time I have encountered more faulty Codegen related to FP8. I'll get back to you with updates ASAP :)
I am converting this PR to draft while I work fleshing it out for more cases. Will provide basic tests and suggested fixes along the way!
@MasterJH5574 @tqchen This should be ready to be reviewed now. I am not 100% familiar with this side of the codebase so please make sure I am not making any silly mistakes and that this doesn't break any other things.
Also feel free to make edits if something is fishy :)
There we go, it should now be good to go! Had some rebase issues, so sorry for spamming updates