aimet
aimet copied to clipboard
quic
CUDA 10 support latest release Pytorch
Dear collaborators,
We have noticed that the latest two releases only support CUDA 11, however some of the computing resources we use are only availble with CUDA 10. Is there an option for backwards compatibility?
Best, Yannick
@struemya Thank you for the query. @quic-akhobare could you comment on this?
Hi @struemya - if you build AIMET code from source, it should build with a 10.x version of CUDA. You will need to install the appropriate versions of PyTorch etc.
If you are using the pre-built AIMET packages, yeah we currently only support 1 version of CUDA at a time. But we are actively looking into having the release build packages for both CUDA 10.2 and CUDA 11. Would that work for your use case?
How to build from source code? I only found the installation instruction from source code in docker. Thx. @quic-akhobare
Hi, @quic-akhobare. I confuse that AIMET is created for accelerate Qualcomm hardware so why do we install CUDA (Nvidia-hardware)?? Best regards,
How to build from source code? I only found the installation instruction from source code in docker. Thx. @quic-akhobare You can use relevant instructions from here https://github.com/quic/aimet/blob/develop/packaging/google_colab/google_colab_development.md#aimet-build-and-installation
You will to install the dependencies. And the "AIMET build and installation" section. Let us know if you get stuck
Hi, @quic-akhobare. I confuse that AIMET is created for accelerate Qualcomm hardware so why do we install CUDA (Nvidia-hardware)??
I think you misunderstood. AIMET is an off-target tool that can be used to optimize models for on-target installation. Since AIMET runs off-target, it can use CUDA to speed up the process of optimizing the models. And you can take the resulting models to other quantized targets as well - not just limited to Qualcomm targets
@quic-akhobare thanks for your explaining, If I have a Qualcomm gpu like Qualcomm AIC, which version of AIMET should I choose to install, AIMET cpu or AIMET gpu?
Closing this issue due to inactivity. Please re-open it/ create a new issue if you need further help.
