Florian Beier
Florian Beier
@yanxxd pinging you again, has this issue been fixed by now or was it invalid?
Thanks! Perhaps this issue should be reopened then.
I have the same problem (Version 4.4.7, Ubuntu 20.10).
I'd like to see this implemented too!
@marty1885 I believe ONNX uses [this](https://github.com/microsoft/onnxruntime/tree/main/onnxruntime/core/providers/vitisai) provider to access the accelerator. Does that help in any way?
@marty1885 if my understanding of the hierarchy of the libraries presented [here](https://www.xilinx.com/developer/articles/introduction-to-vitis-ai-library-apis.html) is correct, then the lowest abstraction is the Xilinx Runtime Library (XRT). Here is the documentation: https://xilinx.github.io/XRT/master/html/index.html Does...
I'm using a Python 3.12 virtual environment on Ubuntu 24.10 and don't have this issue.
I suggest using a virtual environment (`python3 -m venv .venv`). This avoids conflicts with existing python packages and keeps your system clean.
> So wondering what are the VRAM requirement's for 7B model 24 GB (V)RAM are not enough for the 7B model. I was barely able to get it running with...
@justinvforvendetta maybe [this](https://pytorch.org/tutorials/beginner/blitz/data_parallel_tutorial.html) helps.