elkay
elkay
> ``` > [pip3] torchvision==0.16.2+cu121 > [conda] torchvision 0.16.2+cu121 pypi_0 pypi > ``` > > Try uninstalling these versions first? What would that accomplish? That's literally the package that I'm...
> > Built Torch 2.1.2 and TorchVision 2.1.2 from source > > What version of torchvision are you building from source, exactly? There's no torchvision 2.x. The latest stable version...
The box is shut down but I believe it was pyproject.toml that I had to update to point directly at my torch whl and the command I used was "python...
Also interested in this to run in AWS Graviton servers.
Same, but for a custom build to run on AWS Linux. torch.__version__ = 2.3.0a0+git26431db Everything else otherwise works, I just can't get exllamav2 to use flash_attn even if simply installing...
Actually, just noticed this PR. https://github.com/Dao-AILab/flash-attention/pull/757 Seems like exactly what I'm looking for. Hopefully the PR can be approved.
> Does either #757 or #724 work for you? 757 did end up working for me.
Having the same issues. Those files aren't in the tmp folder.
Figured it out, you need to ls the /tmp directory and get the string that represents "*" in the path. Replace "*" with that string and the scripts will run...
Some additional information - After testing a bit further, this error only seems to be happening when using GPTQ format models (my preference due to speed). When I loaded the...