Gaurav Shukla
@stellaraccident @rsuderman This issue arises in the opt-125M model at `%47 = torch.aten.add.Tensor %3, %46, %int1 : !torch.vtensor, !torch.vtensor, !torch.int -> !torch.vtensor` While lowering it to linalg.generic, the dynamic dimensions...
> I think it is likely that the problem is leading up to this and we are not narrowing it to a static value. > > You can't dynamically switch...
> I think it is likely that the problem is leading up to this and we are not narrowing it to a static value. > > You can't dynamically switch...
@xgupta Let me address all the comments of the PR, will wrap it up soon and close this one.
@MaheshRavishankar 1. Following are the commands: ``` ~/iree-build/tools/iree-compile --iree-input-demote-i64-to-i32 --iree-hal-target-backends=llvm-cpu conv2d.linalg.mlir > conv2d.bf16.vmfb 2>iree-compile.log ``` ``` ~/iree-build/tools/iree-run-module --module=conv2d.bf16.vmfb --input="2x8x12x16xbf16=@inference_input.0.bin.txt" > inference.log ``` I am attaching the `inference_input.0.bin`, which is all...
[conv2d.bf16.linalg.mlir.txt](https://github.com/nod-ai/SHARK/files/14340857/conv2d.bf16.linalg.mlir.txt) [conv2d.fp32.linalg.mlir.txt](https://github.com/nod-ai/SHARK/files/14340860/conv2d.fp32.linalg.mlir.txt) [iree-compile-conv2d-bf16.log](https://github.com/nod-ai/SHARK/files/14340866/iree-compile-conv2d-bf16.log) [iree-compile-conv2d-fp32.log](https://github.com/nod-ai/SHARK/files/14340867/iree-compile-conv2d-fp32.log)
Running conv2d with different precisions, keeping all the constants(weight and bias) same. [conv2d.bf16.compile.log](https://github.com/nod-ai/SHARK/files/14359026/conv2d.bf16.compile.log) [conv2d.fp32.compile.log](https://github.com/nod-ai/SHARK/files/14359029/conv2d.fp32.compile.log)
1. Linear Module for reference output: (weight, bias and input has been fixed to simplify comparison in IRs and outputs. Also all these values fits in bf16, so x.to(torch.bfloat16) won't...
> I think this is not really a codegen issue. This is really a bf16 issue . Are we comfortable closing this, or do we need to do more here....
It seems the mismatch is due to different rounding mechanisms used by pytorch and IREE. I ran few simple add/mul tests, and it's mostly 1-bit difference in the outputs. The...