Change the naming scheme of fused instructions to something functional-like

[CharlieL7]

• Currently our fused instructions have names like add_noop_concat_sigmoid. This gives no information about the ordering or the number of inputs into the instruction.
• We could instead rename the fusions into something like

sigmoid(concat(add(0, 1), noop(2)))

This way we have information about the number of inputs and the ordering of the operations.

[pfultz2]

You cant name functions in C++ with parenthesis or spaces, which is why we use underscores as a separator(its also UB to have double underscores as well).

If you want to see the order of operators, you can inspect the submodules before compile_ops is called. This is a better option than trying to encode the graph into a kernel function name, which doesnt even handle everything the graph can represent, like how do you handle when a instruction is used more than once?

[CharlieL7]

If you want to see the order of operators, you can inspect the submodules before compile_ops is called. This is a better option than trying to encode the graph into a kernel function name, which doesnt even handle everything the graph can represent, like how do you handle when a instruction is used more than once?

At the minimum we need a better way of doing this than looking at the pre-compiled submodules. It's not clear which pointwise submodule goes to which compiled op without having to do a bunch of mental comparisons. We shouldn't be using our effort to do mental string comparisons when it is clear to the program what the fused operator does. It would be a better use of engineer time to be looking at higher level things.

[pfultz2]

At the minimum we need a better way of doing this than looking at the pre-compiled submodules. It's not clear which pointwise submodule goes to which compiled op without having to do a bunch of mental comparisons.

What mental comparisons? You just take the module name and jump to its printout, which starts by its name.

Also what is the motivation for this?

Are you trying to see what operators are done to see if they can be rewritten in a better way? For that its best to look at that before fusion, then you dont need to even lookup submodules.

Are you trying to see how a kernel is implemented? Then you probably want to inspect the submodule or even the dumped sources.

In #2471, I added tracing functions to compilation, which could be used to print out info from each kernel(such as submodules etc), this could be used to possibly to printout some debug information, though I dont know exactly how it would look.