Convert Stable Audio Open Small learning path failing when converting ONNX to LiteRT

[knandersen]

Type of issue

• [x] Instructions in the learning path do not work.
• [ ] Spelling error in the learning path.
• [ ] Instructions in the learning path are hard to follow.
• [ ] Instructions to contribute a new learning path are not clear.
• [ ] Other

Describe the issue For the Convert Stable Audio Open Small model to LiteRT learning path, I'm stuck in Step 3 - Converting Model.

So far, I'm able to convert the Conditioners submodule to ONNX. I don't know a whole lot about ONNX, but I ran the resulting model through an ONNX Checker and it appears to be valid.

The issue is with the ONNX → LiteRT step, using onnx2tf. When run both using the supplied scripts/export_conditioners.py and as a standalone command onnx2tf -i conditioners.onnx -o ./conditioners_litert, it fails on both MacOS and Ubuntu, using both Python 3.10 and 3.11. The output from the command is quite long (full log here), but this seems to be the gist of it:

/home/user/dev/stable-audio/.venv/lib/python3.11/site-packages/tf_keras/src/utils/traceback_utils.py:96:1: note: Error code: ERROR_NEEDS_CUSTOM_OPS
<unknown>:0: error: failed while converting: 'main': 
Some ops in the model are custom ops, See instructions to implement custom ops: https://www.tensorflow.org/lite/guide/ops_custom 
Custom ops: BatchMatMulV3
Details:
	tf.BatchMatMulV3(tensor<1x128x3072xf32>, tensor<3072x768xf16>) -> (tensor<1x128x768xf32>) : {adj_x = false, adj_y = false, device = "", grad_x = false, grad_y = false}
	tf.BatchMatMulV3(tensor<1x128x768xf32>, tensor<768x3072xf16>) -> (tensor<1x128x3072xf32>) : {adj_x = false, adj_y = false, device = "", grad_x = false, grad_y = false}
	tf.BatchMatMulV3(tensor<1x128x768xf32>, tensor<768x768xf16>) -> (tensor<1x128x768xf32>) : {adj_x = false, adj_y = false, device = "", grad_x = false, grad_y = false}

<unknown>:0: note: see current operation: 

To Reproduce List the steps to reproduce the behavior: Follow the tutorial

Expected behavior A clear and concise description of what you expected to happen.

Screenshots Screenshot probably not useful, but log from command here: https://pastesio.com/onnx2tf-74

Desktop (please complete the following information):

• OS: Ubuntu 24.04.2 on PC and MacOS 15.5 on MacBook M2

Additional context Add any other context about the problem here.

[knandersen]

It seems like the clue was actually around a failing model optimization, from the log:

Model optimizing started ============================================================
Traceback (most recent call last):
  File "/home/user/dev/stable-audio/.venv/lib/python3.11/site-packages/onnx2tf/onnx2tf.py", line 684, in convert
    result = subprocess.check_output(
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/share/uv/python/cpython-3.11.13-linux-x86_64-gnu/lib/python3.11/subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/share/uv/python/cpython-3.11.13-linux-x86_64-gnu/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['onnxsim', './conditioners.onnx', './conditioners.onnx']' returned non-zero exit status 1.

WARNING: Failed to optimize the onnx file.

When I tried to run onnxsim conditioners.onnx conditioners.onnx as a stand-alone I got this error:

(stable-audio) (base) user@machine-ubuntu:~/dev/stable-audio/ML-examples/kleidiai-examples/audiogen$ onnxsim -i conditioners.onnx conditioners4.onnx
Installing onnxruntime by `/home/user/dev/stable-audio/.venv/bin/python3 -m pip install onnxruntime`, please wait for a moment..
/home/user/dev/stable-audio/.venv/bin/python3: No module named pip
Traceback (most recent call last):
  File "/home/user/dev/stable-audio/.venv/lib/python3.11/site-packages/onnxsim/onnx_simplifier.py", line 19, in <module>
    import onnxruntime as rt  # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'onnxruntime'

After installing onnxruntime, the script now works. Not sure if there's something wrong with my dependency management (I did use uv), but perhaps onnxruntime should be added to deps. Now it works and the script completes

[jasonrandrews]

Thanks for reporting, we will take a look and see if any clarifications are needed.