ONNX Parser

[slntopp]

Hi, i'm trying to get PyArmNN running with ONNX parser, but always end up with Your ArmNN library instance does not support Onnx models parser functionality. Skipped IOnnxParser import..

I tried precompiled library of versions 21.11 and 22.02 (taken from Releases) and was building pyarmnn locally. Despite python swig_generate.py -v does mention ONNX wrapper, and library has libarmnnOnnxParser.so, and built pip package seem to have it all

armnn/python/pyarmnn/dist/pyarmnn-27.0.0/src/pyarmnn/_generated
├── armnn_deserializer_wrap.cpp
├── armnn_onnxparser_wrap.cpp
├── armnn_tfliteparser_wrap.cpp
├── armnn_version_wrap.cpp
├── armnn_wrap.cpp
├── __init__.py
├── pyarmnn_deserializer.py
├── pyarmnn_onnxparser.py
├── pyarmnn.py
├── pyarmnn_tfliteparser.py
└── pyarmnn_version.py

Still not. What am i missing?

Thanks for your help

[MikeJKelly]

Hi @slntopp

Can you tell me what kind of device you are using this on and which binaries you used?

Can I confirm that you were following the guide in the pyarmnn readme?

[slntopp]

Hi, i am trying on x86_64 right now, i tried according binaries from Releases page here on Github(as mentioned, from this two releases: 21.11 and 22.02)

Yes, i was following exactly that readme

[catcor01]

Hello @slntopp,

I believe I may have a solution that will work for you. It would be great if you could run this and get your feedback on if it works or not.

I noticed that when looking at the .so dependencies of the prebuilt binary libarmnnOnnxParser.so file, some of them are pointing to a 'not found' library path.

$ ldd libarmnnOnnxParser.so
	linux-vdso.so.1 (0x00007fff02d61000)
	libarmnn.so.26 => not found
	libprotobuf.so.3.12.0.0 => not found
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fa254a0f000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fa2549f4000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa254802000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fa2546b1000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fa254e71000)

Solution: libarmnn.so.26 and libprotobuf.so.3.12.0.0 should both be located under the same directory (where you extracted the prebuilt binary .tar.gz file).

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<directory_of_above_two_so_files>

Result:

$ ldd libarmnnOnnxParser.so
	linux-vdso.so.1 (0x00007ffce5b8a000)
	libarmnn.so.26 (0x00007f4cb7e6d000)
	libprotobuf.so.3.12.0.0 (0x00007f4cb751b000)
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f4cb7312000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f4cb72f7000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f4cb7105000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f4cb70fd000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f4cb6fae000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f4cb8791000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f4cb6f8b000)

We would very much appreciate your input to ensure it solves the issue and we can proceed to update our documentation if so.

Kind regards, Cathal.

[slntopp]

Hi, sorry for the big delay.

I checked what you suggested, i had the same picture you sent

Before setting LD_LIBRARY_PATH:

ldd armnnlib/libarmnnOnnxParser.so
        linux-vdso.so.1 (0x00007ffec5655000)
        libarmnn.so.27 => not found
        libprotobuf.so.3.12.0.0 => not found
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f5702bc9000)
        libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f5702ba9000)
        libc.so.6 => /usr/lib/libc.so.6 (0x00007f570299d000)
        libm.so.6 => /usr/lib/libm.so.6 (0x00007f57028b6000)
        /usr/lib64/ld-linux-x86-64.so.2 (0x00007f57030a2000)

After:

ldd armnnlib/libarmnnOnnxParser.so
        linux-vdso.so.1 (0x00007ffcf6fa0000)
        libarmnn.so.27 => /home/user/project/armnnlib/libarmnn.so.27 (0x00007f3c40400000)
        libprotobuf.so.3.12.0.0 => /home/user/project/armnnlib/libprotobuf.so.3.12.0.0 (0x00007f3c3fa00000)
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f3c3f7c9000)
        libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f3c40e9d000)
        libc.so.6 => /usr/lib/libc.so.6 (0x00007f3c3f5bd000)
        libdl.so.2 => /usr/lib/libdl.so.2 (0x00007f3c40e96000)
        libm.so.6 => /usr/lib/libm.so.6 (0x00007f3c40b19000)
        /usr/lib64/ld-linux-x86-64.so.2 (0x00007f3c40ee6000)
        libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f3c40e91000)

But that didn't fix the issue, still getting

import pyarmnn as ann
# => Your ArmNN library instance does not support Onnx models parser functionality.  Skipped IOnnxParser import.

I tried with version 21.11 again.

[FrancisMurtagh-arm]

Hi @slntopp, is this still an issue?

At any stage did you install the Debian packages using apt? They don't support ONNX parser so maybe that is what is getting imported.

Regards, Francis.

[FrancisMurtagh-arm]

Closing due to lack of activity, please reopen if issue persists.

Regards, Francis.

[slntopp]

Hi, sorry for delay Yes, it's still an issue No, I didn't use any APT packages nor Debian at all

[FrancisMurtagh-arm]

Hi @slntopp,

After calling python swig_generate.py -v how did you install pyarmnn?

I tried building main via cmake and it worked OK on x86, and I tried with 22.02 binaries.

With the binaries I first made a virtual env set my environment variables to the downloaded binaries as well as the include directories in the repository.

python -m venv env
source env/bin/activate

export  ARMNN_LIB=/path/to/libs
export  LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/libs
export  ARMNN_INCLUDE=/full/path/to/armnn/include:/full/path/to/armnn/profiling/common/include

Then I called python swig_generate.py -v to create the swig wrappers.

After that I called pip install ~/dev/devenv/armnn/python/pyarmnn this points to pyarmnn folder within repository.

This is the output:

Processing ~/dev/devenv/armnn/python/pyarmnn
  Preparing metadata (setup.py) ... done
Requirement already satisfied: numpy in ~/dev/devenv/armnn/env/lib/python3.9/site-packages (from pyarmnn==28.0.0) (1.24.1)
Using legacy 'setup.py install' for pyarmnn, since package 'wheel' is not installed.
Installing collected packages: pyarmnn
  Running setup.py install for pyarmnn ... done
Successfully installed pyarmnn-28.0.0

I could then import pyarmnn successfully with Onnx Parser:

python3
Python 3.9.12 (main, Apr  5 2022, 06:56:58) 
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyarmnn as ann
>>> import imageio
>>> parser = ann.IOnnxParser()
>>> network = parser.CreateNetworkFromBinaryFile('~/dev/devenv/models/mnist_onnx.onnx')
>>> input_binding_info = parser.GetNetworkInputBindingInfo('Input3')
>>> network = parser.CreateNetworkFromBinaryFile('~/dev/devenv/models/mnist_onnx.onnx')
>>> options = ann.CreationOptions()
>>> runtime = ann.IRuntime(options)
>>> preferredBackends = [ann.BackendId('CpuRef')]
>>> opt_network, messages = ann.Optimize(network, preferredBackends, runtime.GetDeviceSpec(), ann.OptimizerOptions())

Regards, Francis.

[matthewsloyanARM]

Hi @slntopp,

We have updated the python/pyarmnn/README.md when using pre-built binaries, as referenced by Francis above. This has been merged to our main branch now, so feel free to try this guide and let us know if it fixes your issue?

Kind regards,

Matthew

[matthewsloyanARM]

Hi @slntopp,

I am going to close this issue as a fix has been merged. If you are still having the issues feel free to reopen the ticket. Thanks again for getting in touch.

Kind regards,

Matthew