The latest Edge TPU runtime (v15.0) possibly breaks old models?

[jingw222]

Running the models against the latest edgetpu runtime gave me an error that previously had been caused by package upgrades.

python simple_pose.py

Traceback (most recent call last):
  File "simple_pose.py", line 25, in <module>
    engine = PoseEngine('models/mobilenet/posenet_mobilenet_v1_075_481_641_quant_decoder_edgetpu.tflite')
  File "/home/pi/WorkingDirectory/opensource/project-posenet/pose_engine.py", line 85, in __init__
    BasicEngine.__init__(self, model_path)
  File "/home/pi/.pyenv/versions/dash_app/lib/python3.7/site-packages/edgetpu/basic/basic_engine.py", line 92, in __init__
    self._engine = BasicEnginePythonWrapper.CreateFromFile(model_path)
RuntimeError: Internal: Unsupported data type in custom op handler: 1006632960Node number 0 (edgetpu-custom-op) failed to prepare.
Failed to allocate tensors.

Here's the setup on my Pi 4 running Raspbian Buster.

libedgetpu1-max:armhf==15.0
pycoral==1.0.0
tflite-runtime==2.5.0

That was not an issue previously under

libedgetpu1-max:armhf==14.1
python3-edgetpu==??
tflite-runtime==2.1.0

which I can't figure out how to downgrade to.

[danieldanuega]

I used python3-edgetpu==2.14.1 with older edgetpu runtime and it worked. I've tried python-edgetpu==2.14.1 with newer runtime and it didn't work at all. I think you need to downgrade the runtime too, but I couldn't find it how to downgrade the runtime, because google change the url.

[danieldanuega]

I am currently trying to update the pose_engine.py with pycoral api, but at the moment I face some difficulty to convert the code. I will do PR after I found the way to convert this.

[jingw222]

I used python3-edgetpu==2.14.1 with older edgetpu runtime and it worked. I've tried python-edgetpu==2.14.1 with newer runtime and it didn't work at all. I think you need to downgrade the runtime too, but I couldn't find it how to downgrade the runtime, because google change the url.

Yes, exactly. For downgrading the runtime module, I think reinstalling python3-edgetpu, the deprecated Python API, would automatically install the legacy version of the runtime package libedgetpu1-legacy-std, which in fact can be install separately alongside libedgetpu1-std. So there's that.

[FHermisch]

Hello everyone, you can make these models run again by following these steps (most of them already mentioned here). This is an excerpt of my personal Wiki ;-) How to load custom delegates if you need them: If a model features an operation which is not part of TFLite nor Libedgetpu, these operations needs to be provided by a custom delegate. This is essentially the case for the 'posenet' model in which the operation to decode the raw inferencing results is embedded as a custom operation.
In the legacy edgetpu-library, the custom delegate for pose-net was loaded within the library. With the new 'libedgetpu' library, it is settled down to the standard TFlite approach of loading a custom delegate. In TFlite, this is done by 'interpreter.load_delegate'.
Currently, there is no Python function (or I havent found one yet) to do this in an intended way, so one has to override the functionality in 'pycoral.utils.edgetpu.make_interpreter' and load the basis delegate for 'libedgetpu' together with all custom delegates:

def make_interpreter(model_path_or_content, device=None, delegate_paths=[]):
    delegs = []
    for dpath in delegate_paths:
        delegs.append( tflite.load_delegate(dpath, {'device': device} if device else {}))
    delegs.append( edgetpu.load_edgetpu_delegate( {'device': device} if device else {}))    
    if isinstance(model_path_or_content, bytes):
        return tflite.Interpreter( model_content=model_path_or_content, experimental_delegates=delegs)
    else:
        return tflite.Interpreter( model_path=model_path_or_content, experimental_delegates=delegs)

This will load the standard Edgetpu delegate and also other delegates from provided paths which point to a compiled library of that delegate (.so / .so.1).
How to compile a custom delegate: The Posenet decoder operation is supplied by Coral as part of the posenet repository: Posenet Repo
It is part of the libcoral library (dont use the old one as I did because it will not work). To get a compiled version of this library, one has to compile it to the needed OS/architecture. This can be done with bazel. First install bazel (I did this on Debian on AMD64 architecture):

sudo apt install curl gnupg
curl -fsSL https://bazel.build/bazel-release.pub.gpg | gpg --dearmor > bazel.gpg
sudo mv bazel.gpg /etc/apt/trusted.gpg.d/
echo "deb [arch=amd64] https://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list
sudo apt update && sudo apt install bazel

We have used version bazel version '3.7.1', which can also be installed dedicated by adding '=3.7.1' after the name 'bazel' in the apt command. The repositories can be cloned with git. The 'libedgetpu' repo needs to be hooked into the libcoral library. As a symbolic link failed for us (too deep directory structure or alike error), we just copied the contents of the 'libedgetpu' library into that directory (this needs to be better set-up, as it will destroy all git functionality). Now, the delegate can be compiled by directing bazel to compile the entry in the respective 'BUILD' file.

sudo bazel build coral/posenet/posenet_decoder.so --experimental_repo_remote_exec

The compiled library will be put in a 'bazel-bin' directory and can now be copied to a place accessible from the python runtime.

[Namburger]

Hi folks, sorry for the late respond here. We're aware that the deprecated API won't be working and is actively working on a fix for this. For now please use my fork for this repo: https://github.com/Namburger/project-posenet Will soon add unit tests and decide on how to fix this official repo.

Thanks and regards!

[Akranar]

Hi @Namburger,

Thanks for providing the fork. I'm trying to run it on a Raspberry Pi 4 with the Coral USB Accelerator, but I'm getting this error:

(venv) pi@raspberrypi:~/posenet_fork $ python simple_pose.py 
Traceback (most recent call last):
  File "simple_pose.py", line 26, in <module>
    poses, inference_time = engine.DetectPosesInImage(pil_image)
  File "/home/pi/posenet_fork/pose_engine.py", line 121, in DetectPosesInImage
    self.run_inference(input_data.flatten())
  File "/home/pi/posenet_fork/pose_engine.py", line 96, in run_inference
    edgetpu.run_inference(self._interpreter, input_data)
  File "/home/pi/posenet_fork/venv/lib/python3.7/site-packages/pycoral/utils/edgetpu.py", line 185, in run_inference
    expected_input_size)
TypeError: InvokeWithMemBuffer(): incompatible function arguments. The following argument types are supported:
    1. (arg0: object, arg1: int, arg2: int) -> None

Invoked with: 15075016, 3008741384, 924963

This was after I had gotten an error that it couldn't find the shared library for the decoder at "posenet_lib/armv7l/posenet_decoder.so" but I saw that the folder for "armv7a" existed, so I just copied that over as "armv7l" for the app to find.

Here's my setup info:

(venv) pi@raspberrypi:~/posenet_fork $ pip freeze
numpy==1.19.4
Pillow==8.0.1
pycairo==1.20.0
pycoral @ https://github.com/google-coral/pycoral/releases/download/release-frogfish/pycoral-1.0.0-cp37-cp37m-linux_armv7l.whl
PyGObject @ file:///home/pi/gstreamer_src/pygobject/PyGObject-3.39.0.dev0-cp37-cp37m-linux_armv7l.whl
tflite-runtime==2.5.0

(venv) pi@raspberrypi:~/posenet_fork $ sudo dpkg --list | grep libedge
ii  libedgetpu1-std:armhf                  15.0                                armhf        Support library for Edge TPU
(venv) pi@raspberrypi:~/posenet_fork $ sudo dpkg --list | grep pycoral
ii  python3-pycoral                        1.0  

Any help or advice you could provide would be greatly appreciated. Thank you.

[schwart-dk]

Hi @Namburger,

Thanks a lot for the fork, which works perfectly on Ubuntu 20.4 on X64 with the Coral Edge USB. However, moving it to to a Raspeberry Pi 4B I run into exactly the same problem as Akranar, above. Should it be ok to use posenet_lib/armv7a/posenet_decoder.so, even if os.uname().machine reports: armv7l?

Any idea where I could try looking?

[manoj7410]

@schwart-dk Can you try with the lib https://github.com/google-coral/project-posenet/blob/master/posenet_lib/armv7l ?

[hjonnala]

Feel free to create new issue if they are any issues with new versions. Thanks!

[google-coral-bot[bot]]

Are you satisfied with the resolution of your issue? Yes No