add PDLL support to iree

[IanWood1]

Note: I messed up merging with main and 5 extra people got added automatically as reviewers, so I fixed the branch and recreated the old PR (https://github.com/iree-org/iree/pull/17097).

Overview

PDL Language (PDLL) is a declarative frontend language for creating pattern rewrites for MLIR. PDLL provides a intuitive interface for creating rewrites. Patterns allow users to declare a sequence of ops to match against and describe how they should be rewritten. PDLL allows for Constraints and Rewriters to either be written within PDLL or written in C++ and exposed through the C++ API. This PR contains examples of both native constraints and native rewriters within the mlp_plugin folder.

mlir-pdll provides functionality for parsing PDLL and generating either C++ or PDL. Adding support for PDLL requires the following functionality from mlir-pdll:

1. Generating a PDLL AST from the input file (parsePDLLAST())
2. Generating a PDL pattern module from the PDLL AST (codegenPDLLToMLIR())

After the PDL module has been constructed, the implementation is nearly identical to ApplyPDLPatterns.cpp, which is already added to iree to support PDL patterns.

PR Summary:

• Added PDLL support to iree via the iree-preprocessing-pdll-spec-filename flag (matching iree-preprocessing-pdl-spec-filename already in place for PDL). This flag allows iree to use a pdll spec in the preprocessing pass.
• Added tests to tosa plugins, torch plugins, and mlp_plugins
• Small update to extensions.md mentioning the new flag
• Added small readme in mlp_plugins that lists the 3 different languages used (transform dialect, pdl, pdll)

---

Since I had to create a new PR here are the requested changes from the old PR:

1. Removed errant file (https://github.com/iree-org/iree/pull/17097#discussion_r1575671594)
2. Removed unnecessary include (https://github.com/iree-org/iree/pull/17097#discussion_r1575672705)
3. Moved PDLL preprocessing pass out of the ApplyPDLPatterns.cpp file and into the new ApplyPDLLPatterns.cpp. Since PDLL relied on functions in the PDL file, I moved the utility functions to Utilities.cpp/h (https://github.com/iree-org/iree/pull/17097#discussion_r1575673162).
4. Created a torch test (https://github.com/iree-org/iree/pull/17097#discussion_r1576364766)

---

byo llvm related stuff

Stella's work on mlir-tblgen PDLL Parser cmake mlir-tblgen lib cmake lib tablegen cmake

[IanWood1]

Currently byo_llvm is failing because llvm isn't being built with any of the PDLL libraries. I think this can be fixed by adding mlir-pdll to LLVM_MLIR_TOOLS in the config file. The mlir-pdll cmake file contains the required dependencies so I think this would solve the issue.

However, as @ScottTodd said on the previous pr (https://github.com/iree-org/iree/pull/17097#discussion_r1572641898), iree doesnt distribute mlir-pdll so there might be issues with that

[MaheshRavishankar]

Currently byo_llvm is failing because llvm isn't being built with any of the PDLL libraries. I think this can be fixed by adding mlir-pdll to LLVM_MLIR_TOOLS in the config file. The mlir-pdll cmake file contains the required dependencies so I think this would solve the issue.

However, as @ScottTodd said on the previous pr (#17097 (comment)), iree doesnt distribute mlir-pdll so there might be issues with that

@ScottTodd is there a problem with doing this... I think its worth doing anyway (mlir-pdll serves the same purpose as mlir-opt, etc. in my book)

[ScottTodd]

Currently byo_llvm is failing because llvm isn't being built with any of the PDLL libraries. I think this can be fixed by adding mlir-pdll to LLVM_MLIR_TOOLS in the config file. The mlir-pdll cmake file contains the required dependencies so I think this would solve the issue. However, as @ScottTodd said on the previous pr (#17097 (comment)), iree doesnt distribute mlir-pdll so there might be issues with that

@ScottTodd is there a problem with doing this... I think its worth doing anyway (mlir-pdll serves the same purpose as mlir-opt, etc. in my book)

Can you start by adding a brief overview of what PDLL is at the top of the PR description? I can't make much of a judgement from just looking at the details here and want to zoom out first. Referencing https://mlir.llvm.org/docs/PDLL/ could help too.

[IanWood1]

It's probably okay to close it.

There are issues with byo llvm and PDLL also doesn't support ops with regions (e.g. linalg named ops), which makes it fairly limited in its usefulness. PDLL can be used at compile time to generate C++ patterns, but I'm not sure if there are any usecases.