onnx-mlir
onnx-mlir copied to clipboard
onnx
Modify RunONNXModel.py for pip install
The goal is to create a python package so that we can run an inference like onnxruntime in python script. Since the RunONNXModel.py provides most of the necessary functionality, I added extra support in that script. The pip package related code will be checked later. I tested with a local pip install. We can upload the package later.
Major changes:
- Add the function interface to call the main in RunONNXModel.py with the parameters similar to onnxruntime:
- Accept list of arrays as input.
- Return the outputs
Example:
import numpy as np
import onnxmlirrun
a = np.zeros((3,4,5), dtype=np.float32)
b = a + 4
r = onnxmlirrun.RunONNXModel.onnxmlirrun(compiled_so="/Users/chentong/Projects/onnx-mlir/build/test_add.so", inputs=[a,b])
print(r)
r = onnxmlirrun.RunONNXModel.onnxmlirrun(onnx_model="/Users/chentong/Projects/onnx-mlir/build/test_add.onnx", inputs=[a,b])
print(r)
We may have different name for the functions and the package. Comments are welcome.
@chentong319 Can you add a reference to the ORT interface that this PR is imitating.
In your first example
r = onnxmlirrun.RunONNXModel.onnxmlirrun(compiled_so="/Users/chentong/Projects/onnx-mlir/build/test_add.so", inputs=[a,b])
was the onnx_model parameter omitted on purpose? Just for me to understand better the interface.
@chentong319 Can you add a reference to the ORT interface that this PR is imitating.
In your first example
r = onnxmlirrun.RunONNXModel.onnxmlirrun(compiled_so="/Users/chentong/Projects/onnx-mlir/build/test_add.so", inputs=[a,b])was the
onnx_modelparameter omitted on purpose? Just for me to understand better the interface.
Yes, onnx_model is not needed if the compiled .so is provided. By the way, the two onnxmilrrun.RunONNXModel.onnxmlirrun commands are two independent ways to do inference.
Got it, so we can provide a onnx file or a pre-compiled binary, smart, thanks.
onnxmlirrun.RunONNXModel.onnxmlirrun
My two cents here. For the package name (the first onnxmlirrun), just onnxmlir is enough to avoid typing two Rs.
I would remove RunONNXModel, but it seems not straightforward unless we reorganize RunONNXModel.py.
Anyway, we have several python utilities, perhaps it's time to reorganize them into a single python package.
I changed the init.py for the package to simplify onnxmlirrun.RunONNXModel.onnxmlirrun to onnxmlir.onnxmlir.
I am thinking to add the code for package into onnx-mlir in future.
Can you copy/link a description of the ORT runtime interface and update ours with the new names, so that we can see how they relate?
Is it something on this page? https://onnxruntime.ai/docs/get-started/with-python.html
The Ort code is like
import onnxruntime as ort
import numpy as np
x, y = test_data[0][0], test_data[0][1]
ort_sess = ort.InferenceSession('fashion_mnist_model.onnx')
outputs = ort_sess.run(None, {'input': x.numpy()})
Since the current RunModelPython does not separate the session creation and run, our code will be like:
import onnxmlir
import numpy as np
x, y = test_data[0][0], test_data[0][1]
outputs = onnxmlir.run(onnx_model='fashion_mnist_model.onnx', inputs = [a, b])
It is doable to totally mimic ORT in separating the session creation and run, and using dictionary for input.
Ideally
import onnxmlir as ort
import numpy as np
x, y = test_data[0][0], test_data[0][1]
ort_sess = ort.InferenceSession('fashion_mnist_model.onnx')
outputs = ort_sess.run(None, {'input': x.numpy()})
with additional parameters ok
import onnxruntime as ort
import numpy as np
x, y = test_data[0][0], test_data[0][1]
ort_sess = ort.InferenceSession('fashion_mnist_model.onnx', compile_flags="-O3")
outputs = ort_sess.run(None, {'input': x.numpy()})
reusing flags that already exists, if any.
Check error message if file does not exist.
onnx-mlir will report the error if file does not exist. Leave this to the TODO list to catch all the errors gracefully.
