iree icon indicating copy to clipboard operation
iree copied to clipboard

Summarize dispatch op names using linalg interfaces and heuristics.

Open ScottTodd opened this issue 3 years ago • 1 comments

Fixes https://github.com/iree-org/iree/issues/7211 (tags could still include information like runtime dynamic dimension values though): image

This uses heuristics to summarize the "most interesting" or "highest computational cost" component of each dispatch function (executable entry point). Once a summary is generated, it is appended to the function name, so main_dispatch_123 becomes something like main_dispatch_123_matmul_384x512x384.

Tools like Tracy will show the new function names, which should help developers see how parts of a program are performing at a glance, without needing to cross reference with the source or intermediate .mlir output.

Executable names themselves are left unchanged, so linked (mobilebertsquad.tflite_linked_llvm) and unlinked (static linking sample uses simple_mul_dispatch_0_library_query) names are not dependent on the particular heuristics used. Note that this also means that the file names generated by --iree-hal-dump-executable-sources-to={path} will not include the new suffixes.


We can extend the heuristics over time, such as by

  • including input/output element types (i8, i32, f32, etc.)
  • adding handling for LinalgExtOp
  • improving the cost model
  • marking if fusion was performed

Running with --debug-only=iree-dispatch produces output like

//--- summarizing 'main_dispatch_5' ---//
// linalg.fill cost: 196608
// new best op; summary: 'fill_384x512', cost: 196608
// linalg.matmul cost: 75497472
// new best op; summary: 'matmul_384x512x384', cost: 75497472
// linalg.generic cost: 196608
//--- opSummary: 'matmul_384x512x384' ---//

(see the full output for mobilebert here)

ScottTodd avatar Aug 10 '22 19:08 ScottTodd

@okkwon do you want to review this also? (you filed a similar feature request in https://github.com/iree-org/iree/issues/9132)

ScottTodd avatar Aug 11 '22 21:08 ScottTodd