[Tracking Issue] UMA: Universal Modular Accelerator Interface

[MichaelJKlaiber]

This is a global tracking issue for landing UMA. The original RFC can be found here. Please feel free to bring in more questions to the corresponding thread.

To make the improvement manageable, we will be breaking the code into several steps. The main architecture scaffolding and data structure changes will come first, then we will upstream individual key features.

Steps

[UMA-0] RFC

• [x] Definition of UMA-RFCv1 https://github.com/apache/tvm-rfcs/pull/60

[UMA-1] Implementation of core infrastructure

Implementation of UMA Backend according to UMA-RFCv1

• [ ] UMA Partitioner https://github.com/apache/tvm/pull/12087
• [ ] UMA Pipeline https://github.com/apache/tvm/pull/12087
• [ ] UMA Lower https://github.com/apache/tvm/pull/12087
• [ ] UMA Codegen https://github.com/apache/tvm/pull/12087
• [ ] Unit tests https://github.com/apache/tvm/pull/12087

[UMA-2] Prototype of UMA using NN accelerator

• [ ] Using UMA for deployment on UltraTrail NN accelerator (University of Tübingen, Germany)
• [ ] Using UMA for deployment on NN accelerator_2 (to be defined)

[UMA-3] Formulation of lessons learned for UMA_v1.0

• [ ] Lessons learned and future topics document based on [UMA-2]
• [ ] move OutlineCompilerFunctionsMutator out of UMA
• [ ] extend/add AOTTestRunner by project API
• [ ] annotations of UMALower._operator_strategies
• [ ] Add test case importing tflite model from .tflite file
• [ ] Add test case importing onnx model from .onnx file

[UMA-4] Documentation

• [ ] Vanilla Accelerator Tutorial https://github.com/apache/tvm/pull/12087
• [ ] Strawberry Accelerator Tutorial
• [ ] Chocolate Accelerator Tutorial

[UMA-5] Suggestion from UMAv1 PR

• [ ] Add UMA registration to tvmc
• [ ] run BERT with UMA on ARM EthosU
• [ ] Case where we lower some parts with UMA while handling other unsupported parts via the conventional pipeline --> Integrate MobileNet unit test case
• [ ] Tutorial as RST for gallery https://github.com/apache/tvm/pull/12087

Working branch

https://github.com/MichaelJKlaiber/tvm/tree/uma

[gfvvz]

Hi, thanks for the work. Just want to know if there is a rough schedule on these features development?

[MichaelJKlaiber]

Hi @gfvvz , that's a great question. I'll try to come up with an answer asap. If you have input or feedback, feel free to reach out any time via Discord, GitHub or the TVM community forum

[MichaelJKlaiber]

@gfvvz , just added the link to our working branch, feel free to peek into (partially finished) functions.

[xinetzone]

python/tvm/relay/backend/contrib/uma/backend.py has a bug：

for name, attr in self._target_attrs:
            if attr is None:
                raise ValueError("Target attribute None is not supported.")

can use:

for attr in self._target_attrs.values():
            if attr is None:
                raise ValueError("Target attribute None is not supported.")

[areusch]

@xinetzone do you mind filing a separate issue and including some more detail (i.e. is there a stacktrace)?

[fpedd]

@xinetzone @areusch there is a PR for that already #12731

[areusch]

oh yes, thanks for the reminder @fabianpedd :)

[qzylalala]

Hi, thanks for the work. I wonder whether this issue is still active? I sincerely hope that there are some progress in ([UMA-2] Prototype of UMA using NN accelerator) ！