Alastair D'Silva

Commenting out Coral4 but leaving Coral3 fails: ``` [s6-init] making user provided files available at /var/run/s6/etc...exited 0. [s6-init] ensuring user provided files have correct perms...exited 0. [fix-attrs.d] applying ownership &...

Could we crowdsource funds for training against llama70B? I'd be happy to contribute

Thanks! I was just about to start work on a similar PR. I'm testing it now. I think it would make more sense to use the source from the current...

Unfortunately, it looks like testing failed: ``` #19` 58.41 [2/2] /usr/local/cuda/bin/nvcc -I/usr/local/lib/python3.10/dist-packages/torch/include -I/usr/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.10/dist-packages/torch/include/TH -I/usr/local/lib/python3.10/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.10 -c -c /build/quant_cuda_kernel.cu -o /build/build/temp.linux-x86_64-3.10/quant_cuda_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H...

It looks like it wants this patch: https://github.com/qwopqwop200/GPTQ-for-LLaMa/pull/58 Bumping the GPTQ SHA to 841feedde876785bc8022ca48fd9c3ff626587e2 gets past this

Yup, my oldest card is an M40, which requires that patch. On 25 March 2023 8:07:28 pm AEDT, loeken ***@***.***> wrote: >@deece tried setting the specific TORCH_CUDA_ARCH_LIST in the docker-compose...

Tesla M40. I also have a Tesla K80, but it doesn't really get used. On 25 March 2023 8:14:30 pm AEDT, loeken ***@***.***> wrote: >@deece with M40 do you mean...

I don't think that will work, as the patch mentioned above suggests that it will break for anything under 6.0. That patch does work though, and ask that is needed...

Thanks, I'm out all day tomorrow, but I'll have another crack on Monday -- Sent from my Android device with K-9 Mail. Please excuse my brevity.

It might be worth squashing/refactoring the commits before merging the PR. Maybe even squashing it down to a single commit?