Deployment on edge devices

[ztianlin]

I want to ask if there is an official practice of how to deploy PI0.5 on edge devices such as Nvidia Jetson Orin & Thor series? Many Thanks.

[Anish2811]

• Use Nvidia reference carrier boards or design per the Jetson Thor/ Orin design guide.
• Ensure proper power sequencing, thermal cooling and sensor interfaces (MIPI CSI, Ethernet, PCIe).
• Deploy software with Jetpack, TensorRT, DeepStream, Issac/Holoscan and optimize models(FP8/FP4).
• Implement secure boot, OTA updates and monitoring for fleet deployment.

[GaohaoZhou-ops]

I want to ask if there is an official practice of how to deploy PI0.5 on edge devices such as Nvidia Jetson Orin & Thor series? Many Thanks.

Hi, our team has tried deploying on Thor. Although there are still issues with Jax and CUDA 13, we have successfully run the examples/simple_client/main.py file. We will do our best to resolve the issues mentioned above. If you are interested, you can check out the full deployment blog. Unfortunately, this blog is written in Chinese and is not yet available in English. We recommend using your browser's translation function. If you encounter any problems, please feel free to ask me.

• Blog：https://blog.csdn.net/nenchoumi3119/article/details/151928137?spm=1001.2014.3001.5501

[zhouyc1006]

I want to ask if there is an official practice of how to deploy PI0.5 on edge devices such as Nvidia Jetson Orin & Thor series? Many Thanks.

Hi, our team has tried deploying on Thor. Although there are still issues with Jax and CUDA 13, we have successfully run the examples/simple_client/main.py file. We will do our best to resolve the issues mentioned above. If you are interested, you can check out the full deployment blog. Unfortunately, this blog is written in Chinese and is not yet available in English. We recommend using your browser's translation function. If you encounter any problems, please feel free to ask me.

• Blog：https://blog.csdn.net/nenchoumi3119/article/details/151928137?spm=1001.2014.3001.5501

RuntimeError: CUDA error: no kernel image is available for execution on the device CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1 Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

Thank you for your work, but I encountered the above issues while configuring on Orin following similar steps as yours. Have you encountered these problems before? How were they resolved? Thanks in advance!

[GaohaoZhou-ops]

I want to ask if there is an official practice of how to deploy PI0.5 on edge devices such as Nvidia Jetson Orin & Thor series? Many Thanks.

Hi, our team has tried deploying on Thor. Although there are still issues with Jax and CUDA 13, we have successfully run the examples/simple_client/main.py file. We will do our best to resolve the issues mentioned above. If you are interested, you can check out the full deployment blog. Unfortunately, this blog is written in Chinese and is not yet available in English. We recommend using your browser's translation function. If you encounter any problems, please feel free to ask me.

• Blog：https://blog.csdn.net/nenchoumi3119/article/details/151928137?spm=1001.2014.3001.5501

RuntimeError: CUDA error: no kernel image is available for execution on the device CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1 Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

Thank you for your work, but I encountered the above issues while configuring on Orin following similar steps as yours. Have you encountered these problems before? How were they resolved? Thanks in advance!

Unfortunately, our previous attempts to deploy the pi0 model on Orin failed. The core issue is that even Orin's 64GB of hardware wasn't powerful enough to support inference. Furthermore, you can't use this blog post for deployment on Orin because it's designed for Thor hardware. Thor uses the Nvidia Blackwell architecture, which differs significantly from Orin's Ampere architecture in terms of CUDA and Torch compilation. If you truly need to run the pi0 model on Orin, you may need to perform additional work related to quantization and compilation.

[GaohaoZhou-ops]

Hello, our team has successfully deployed the pi0.5 model on a Jetson Orin 64GB DK + JetPack 6, with a single inference time of approximately 1 second. Here is our blog link; we hope it helps you. Unfortunately, we currently have no plans to write an English version of the blog. You can use your browser's translation function. If you encounter any problems, you can leave a message here.

https://blog.csdn.net/nenchoumi3119/article/details/154258492?spm=1001.2014.3001.5502