alpaca-lora
alpaca-lora copied to clipboard
tloen
Issue: ERROR: Your GPU does not support Int8 Matmul!
My GPU is NVIDIA RTX A6000, why it remind me my GPU does not support Int8 Matmul! Anyone else encountered this error?
This is the output of running:
Loading checkpoint shards: 100%|████████████████████████████████████| 33/33 [00:13<00:00, 2.49it/s]
Downloading (…)neration_config.json: 100%|█████████████████████████| 124/124 [00:00<00:00, 1.22MB/s]
Downloading tokenizer.model: 100%|███████████████████████████████| 500k/500k [00:00<00:00, 11.9MB/s]
Downloading (…)cial_tokens_map.json: 100%|███████████████████████| 2.00/2.00 [00:00<00:00, 24.2kB/s]
Downloading (…)okenizer_config.json: 100%|█████████████████████████| 141/141 [00:00<00:00, 1.19MB/s]
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization.
The tokenizer class you load from this checkpoint is 'LLaMATokenizer'.
The class this function is called from is 'LlamaTokenizer'.
Found cached dataset json (/home/fqw5095/.cache/huggingface/datasets/yahma___json/yahma--alpaca-cleaned-5d24553f76c14acc/0.0.0/fe5dd6ea2639a6df622901539cb550cf8797e5a6b2dd7af1cf934bed8e233e6e)
100%|████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 233.50it/s]
trainable params: 16777216 || all params: 6755192832 || trainable%: 0.24836028248556738
0%| | 0/3880 [00:00<?, ?it/s]
=============================================
ERROR: Your GPU does not support Int8 Matmul!
=============================================
python: /mmfs1/gscratch/zlab/timdettmers/git/bitsandbytes/csrc/ops.cu:428: int igemmlt(cublasLtHandle_t, int, int, int, const int8_t*, const int8_t*, void*, float*, int, int, int) [with int FORMATB = 4; int DTYPE_OUT = 32; int SCALE_ROWS = 0; cublasLtHandle_t = cublasLtContext*; int8_t = signed char]: Assertion `false' failed.
Aborted (core dumped)
