Kevin Stephano

@pytorchbot merge -g

Comments from slack thread: J Jiang [42 minutes ago](https://pytorch-lightning.slack.com/archives/C04N128ARJ6/p1675794056810389?thread_ts=1675793700.035419&cid=C04N128ARJ6) Because our tensor rank is instantiated via template constant argument. So there's a physical limit to that. Naoya Maruyama [40 minutes...

Ivan Reports: > It should also print to Python console output, not the terminal. In Colab fusion.print() is visible only in the logs, not the usual cell output. Make sure...

It was suggested that we think about iterating over the available kernels for a particular fusion. That is, currently, not possible given the opaque nature of the `FusionExecutorCache` to the...

Christian has mentioned that printing kernels and scheduled IRs is likely gated on effort by @mmigdal-nv to expose printing the kernels from segments instead of dropping in an environment variable.

Are we sure the math issue isn't an indicator that something else is broken given we consciously decided not to implement FP16 math ops in the runtime as they should...

Sounds like Jie is going to look into this in the frontend.

@googlebot I signed it!

Repro of permute based fusion: ``` import torch from torch._C._nvfuser import Fusion, FusionDefinition, DataType def nvfuser_fusion_id0(fd : FusionDefinition) -> None : T0 = fd.define_tensor(symbolic_sizes=[-1, -1, -1], contiguous=[True, True, True], dtype=DataType.Float)...

Here is a second example that should be easier as the second view undoes the first view and I would start here. This is a Gelu operation from the Feed...