raspberry-pi-pcie-devices
raspberry-pi-pcie-devices copied to clipboard
geerlingguy
ASUS AI Accelerator PCIe Card
Hey Jeff, (@geerlingguy)
Have you thought of trying the Asus AI card in Pi's 1x PCIe slot?
https://iot.asus.com/products/AI-accelerator/AI-Accelerator-PCIe-Card/
I assume ASUS marketing should be able to provide you with a review sample.
Depending on the AI/ML workload, an 8GB Pi with a 1GbE connection should be able to keep this card fed. If so, then it would make much more sense to use a Pi in conjunction with this card for edge analytics rather than a 1U Xeon or EPYC server driving 1-4 of these. The same way you can theoretically drive a GPU mining farm using a Pi.
Cheers, Bryan
If the card uses Coral TPUs, then we've already seen that the driver doesn't yet work (unfortunately). If we can get the driver working, it would be great to try one of these.
Oh noes! Yea it's 8/16 Coral M.2 TPUs attached via a PCIe Switch. Now that PiShop has some CM4s back in stock, I was going to try pair a Pi with an accelerator card to get Home Assistant's facial recognition engine running.
I just read your comments on the Coral M.2 about it working for other ARM64 CPUs (i.e. Ampere Altra) but not the Pi, which was all sorts of funny. I don't completely understand exactly what part of the Pi's PCIe implementation is causing the issue, and I'm not sure if the presence of the switch helps or not. Maybe when my Pis arrive I can try pairing it with an RTX card, at least to get access to the CUDA cores.
@bryanvaz - See some of the GPU threads linked here: https://pipci.jeffgeerling.com/#gpus-graphics-cards — so far it's been difficult getting any GPU drivers working on the Pi's PCIe bus. Though @coreforge has had some success with older generation Radeon cards (pre-AMDGPU era). Nvidia's drivers are more of a black box, and nouveau is tough to work with for things like CUDA/AI (and still doesn't work AFAICT).
Ok reading some of the history of your attempts to debug the Nvidia cards is giving me traumatizing flashbacks to my engineering class where we had to build a GPU pipeline on an FPGA ... and then write drivers for it.
Anyways the secret to getting higher than an 80% grade in that class was to fake or ignore data if the pipeline got too slow or overflowed the memory because the TA dumped a massive set of vertices in the pipeline. If the issue is partly tied to the 32-bit PCIe address space we may be able fake the other half, either at the kernel level, or at the driver level. I won't know till I get a Pi in hand.
I don't know why, but this is still not enticing me to switch to a Jetson. I feel like that it would be a full scale development effort getting Home Assistant and face detection to work on the Jeson.
The issue is less tied to address space, and more to the fact that the pi can't handle 64bit long transactions over pcie. If a card requires those, it won't work. It might be possible to put an fpga in between the card and the pi and have some way to tell it to do 64bit transactions, but I'm not sure if that's really feasable.
