Huatuo-Llama-Med-Chinese
Huatuo-Llama-Med-Chinese copied to clipboard
SCIR-HI
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0! (when checking argument for argument index in method wrapper_CUDA__index_select)
/slurm/home/yrd/shaolab/daiyizheng/.conda/envs/llama/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: /slurm/home/yrd/shaolab/daiyizheng/.conda/envs/llama did not contain libcudart.so as expected! Searching further paths...
warn(msg)
/slurm/home/yrd/shaolab/daiyizheng/.conda/envs/llama/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/run/user/80104/vscode-git-4b808d81bf.sock')}
warn(msg)
/slurm/home/yrd/shaolab/daiyizheng/.conda/envs/llama/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/run/user/80104/vscode-ipc-0b328df5-e364-494a-b230-9f7e99271b5b.sock')}
warn(msg)
/slurm/home/yrd/shaolab/daiyizheng/.conda/envs/llama/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('() { eval /usr/bin/modulecmd bash $*\n}')}
warn(msg)
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization.
The tokenizer class you load from this checkpoint is 'LLaMATokenizer'.
The class this function is called from is 'LlamaTokenizer'.
The model weights are not tied. Please use the tie_weights method before using the infer_auto_device function.
Loading checkpoint shards: 0%| | 0/81 [00:00<?, ?it/s]
Loading checkpoint shards: 1%| | 1/81 [00:00<01:08, 1.16it/s]
Loading checkpoint shards: 2%|▏ | 2/81 [00:01<01:11, 1.11it/s]
Loading checkpoint shards: 4%|▎ | 3/81 [00:02<01:13, 1.07it/s]
Loading checkpoint shards: 5%|▍ | 4/81 [00:03<01:10, 1.09it/s]
Loading checkpoint shards: 6%|▌ | 5/81 [00:04<01:09, 1.09it/s]
Loading checkpoint shards: 7%|▋ | 6/81 [00:05<01:09, 1.08it/s]
Loading checkpoint shards: 9%|▊ | 7/81 [00:06<01:10, 1.05it/s]
Loading checkpoint shards: 10%|▉ | 8/81 [00:07<01:07, 1.08it/s]
Loading checkpoint shards: 11%|█ | 9/81 [00:08<01:08, 1.05it/s]
Loading checkpoint shards: 12%|█▏ | 10/81 [00:09<01:06, 1.07it/s]
Loading checkpoint shards: 14%|█▎ | 11/81 [00:10<01:05, 1.06it/s]
Loading checkpoint shards: 15%|█▍ | 12/81 [00:11<01:04, 1.08it/s]
Loading checkpoint shards: 16%|█▌ | 13/81 [00:12<01:02, 1.08it/s]
Loading checkpoint shards: 17%|█▋ | 14/81 [00:13<01:02, 1.07it/s]
Loading checkpoint shards: 19%|█▊ | 15/81 [00:14<01:03, 1.05it/s]
Loading checkpoint shards: 20%|█▉ | 16/81 [00:15<01:02, 1.04it/s]
Loading checkpoint shards: 21%|██ | 17/81 [00:16<01:02, 1.02it/s]
Loading checkpoint shards: 22%|██▏ | 18/81 [00:16<01:01, 1.03it/s]
Loading checkpoint shards: 23%|██▎ | 19/81 [00:17<01:00, 1.03it/s]
Loading checkpoint shards: 25%|██▍ | 20/81 [00:18<00:59, 1.02it/s]
Loading checkpoint shards: 26%|██▌ | 21/81 [00:19<00:58, 1.02it/s]
Loading checkpoint shards: 27%|██▋ | 22/81 [00:20<00:57, 1.02it/s]
Loading checkpoint shards: 28%|██▊ | 23/81 [00:21<00:57, 1.01it/s]
Loading checkpoint shards: 30%|██▉ | 24/81 [00:22<00:55, 1.04it/s]
Loading checkpoint shards: 31%|███ | 25/81 [00:23<00:52, 1.07it/s]
Loading checkpoint shards: 32%|███▏ | 26/81 [00:24<00:51, 1.07it/s]
Loading checkpoint shards: 33%|███▎ | 27/81 [00:25<00:50, 1.07it/s]
Loading checkpoint shards: 35%|███▍ | 28/81 [00:26<00:49, 1.06it/s]
Loading checkpoint shards: 36%|███▌ | 29/81 [00:27<00:48, 1.08it/s]
Loading checkpoint shards: 37%|███▋ | 30/81 [00:28<00:47, 1.07it/s]
Loading checkpoint shards: 38%|███▊ | 31/81 [00:29<00:45, 1.09it/s]
Loading checkpoint shards: 40%|███▉ | 32/81 [00:30<00:45, 1.07it/s]
Loading checkpoint shards: 41%|████ | 33/81 [00:31<00:44, 1.09it/s]
Loading checkpoint shards: 42%|████▏ | 34/81 [00:31<00:42, 1.10it/s]
Loading checkpoint shards: 43%|████▎ | 35/81 [00:32<00:42, 1.09it/s]
Loading checkpoint shards: 44%|████▍ | 36/81 [00:33<00:40, 1.10it/s]
Loading checkpoint shards: 46%|████▌ | 37/81 [00:34<00:38, 1.13it/s]
Loading checkpoint shards: 47%|████▋ | 38/81 [00:35<00:38, 1.12it/s]
Loading checkpoint shards: 48%|████▊ | 39/81 [00:36<00:37, 1.12it/s]
Loading checkpoint shards: 49%|████▉ | 40/81 [00:37<00:37, 1.08it/s]
Loading checkpoint shards: 51%|█████ | 41/81 [00:38<00:36, 1.11it/s]
Loading checkpoint shards: 52%|█████▏ | 42/81 [00:39<00:34, 1.11it/s]
Loading checkpoint shards: 53%|█████▎ | 43/81 [00:40<00:34, 1.10it/s]
Loading checkpoint shards: 54%|█████▍ | 44/81 [00:41<00:34, 1.08it/s]
Loading checkpoint shards: 56%|█████▌ | 45/81 [00:42<00:33, 1.08it/s]
Loading checkpoint shards: 57%|█████▋ | 46/81 [00:43<00:33, 1.06it/s]
Loading checkpoint shards: 58%|█████▊ | 47/81 [00:43<00:31, 1.07it/s]
Loading checkpoint shards: 59%|█████▉ | 48/81 [00:44<00:30, 1.09it/s]
Loading checkpoint shards: 60%|██████ | 49/81 [00:45<00:29, 1.08it/s]
Loading checkpoint shards: 62%|██████▏ | 50/81 [00:46<00:29, 1.07it/s]
Loading checkpoint shards: 63%|██████▎ | 51/81 [00:47<00:27, 1.07it/s]
Loading checkpoint shards: 64%|██████▍ | 52/81 [00:48<00:27, 1.05it/s]
Loading checkpoint shards: 65%|██████▌ | 53/81 [00:49<00:27, 1.03it/s]
Loading checkpoint shards: 67%|██████▋ | 54/81 [00:50<00:25, 1.04it/s]
Loading checkpoint shards: 68%|██████▊ | 55/81 [00:51<00:26, 1.00s/it]
Loading checkpoint shards: 69%|██████▉ | 56/81 [00:52<00:24, 1.02it/s]
Loading checkpoint shards: 70%|███████ | 57/81 [00:53<00:23, 1.02it/s]
Loading checkpoint shards: 72%|███████▏ | 58/81 [00:54<00:22, 1.04it/s]
Loading checkpoint shards: 73%|███████▎ | 59/81 [00:55<00:20, 1.06it/s]
Loading checkpoint shards: 74%|███████▍ | 60/81 [00:56<00:19, 1.06it/s]
Loading checkpoint shards: 75%|███████▌ | 61/81 [00:57<00:19, 1.05it/s]
Loading checkpoint shards: 77%|███████▋ | 62/81 [00:58<00:18, 1.05it/s]
Loading checkpoint shards: 78%|███████▊ | 63/81 [00:59<00:17, 1.04it/s]
Loading checkpoint shards: 79%|███████▉ | 64/81 [01:00<00:15, 1.06it/s]
Loading checkpoint shards: 80%|████████ | 65/81 [01:01<00:14, 1.07it/s]
Loading checkpoint shards: 81%|████████▏ | 66/81 [01:01<00:13, 1.08it/s]
Loading checkpoint shards: 83%|████████▎ | 67/81 [01:02<00:12, 1.10it/s]
Loading checkpoint shards: 84%|████████▍ | 68/81 [01:03<00:12, 1.08it/s]
Loading checkpoint shards: 85%|████████▌ | 69/81 [01:04<00:10, 1.10it/s]
Loading checkpoint shards: 86%|████████▋ | 70/81 [01:05<00:10, 1.09it/s]
Loading checkpoint shards: 88%|████████▊ | 71/81 [01:06<00:09, 1.07it/s]
Loading checkpoint shards: 89%|████████▉ | 72/81 [01:07<00:08, 1.09it/s]
Loading checkpoint shards: 90%|█████████ | 73/81 [01:08<00:07, 1.05it/s]
Loading checkpoint shards: 91%|█████████▏| 74/81 [01:09<00:06, 1.07it/s]
Loading checkpoint shards: 93%|█████████▎| 75/81 [01:10<00:05, 1.08it/s]
Loading checkpoint shards: 94%|█████████▍| 76/81 [01:11<00:04, 1.09it/s]
Loading checkpoint shards: 95%|█████████▌| 77/81 [01:12<00:03, 1.08it/s]
Loading checkpoint shards: 96%|█████████▋| 78/81 [01:13<00:02, 1.05it/s]
Loading checkpoint shards: 98%|█████████▊| 79/81 [01:14<00:01, 1.06it/s]
Loading checkpoint shards: 99%|█████████▉| 80/81 [01:15<00:00, 1.06it/s]
Loading checkpoint shards: 100%|██████████| 81/81 [01:15<00:00, 1.21it/s]
Loading checkpoint shards: 100%|██████████| 81/81 [01:15<00:00, 1.07it/s]
Traceback (most recent call last):
File "/slurm/home/yrd/shaolab/daiyizheng/nlp/Huatuo-Llama-Med-Chinese/infer.py", line 132, in