js-2024

I'm having the same issue on Linux Mint with 7x3090. The behavior is almost identical to what diabeticpilot described above, right down to the GPUs loading up a little under...

I'm getting the same 'transformers.masking_utils' error when I try to load. no balance_serve W0930 16:06:02.902000 115051 site-packages/torch/utils/cpp_extension.py:2425] TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation....