q5sys (JT)
q5sys (JT)
I noticed the other day when I was attempting to quantize the 34B larger models that the Granite models are 2 different types. The 3B,7B, and 8B models are llama,...
+1 on this, this is one of those small improvements I've been hoping would come for a while.
Sorry, but that's not possible, Nvidia doesn't support CUDA on FreeBSD.
That is using the linuxulator compatibility layer and is not natively supported. Also there's been updates to the both the compat layer and the tools since that was written, as...
This would be helpful.
Initial install needs to be done with internet access, but I've been able to use it offline after that. I dont think it needs that laion model unless you enable...
@cmdr2 Setup tools needs to be
For anyone that wants a way to hack this to work until it's fixed. Download the non broken setuptools package here: https://anaconda.org/conda-forge/setuptools/69.5.1/download/noarch/setuptools-69.5.1-pyhd8ed1ab_0.conda Unzip that somewhere and open up the `pkg-setuptools-69.5.1-pyhd8ed1ab_0.tar.zst`...
@SenkiJun Hmmm, wonder why it didn't work for you. I just did it again in a fresh path on my Rocky box and it worked. IDK why Arch would be...
> @q5sys I probably swapped out the files too late. Thanks for your time and help. I think this will only work a fresh install, I don't think it will...