Baichuan-13B icon indicating copy to clipboard operation
Baichuan-13B copied to clipboard

多卡运行demoRuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cuda:1!

Open Young1993 opened this issue 1 year ago • 0 comments

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cuda:1! Traceback: File "/home/kemove/anaconda3/envs/instructor/lib/python3.10/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 552, in _run_script exec(code, module.dict) File "/home/kemove/shimin/instructor-embedding/baichuan.py", line 83, in main() File "/home/kemove/shimin/instructor-embedding/baichuan.py", line 74, in main for response in model.chat(tokenizer, messages, stream=True): File "/home/kemove/.cache/huggingface/modules/transformers_modules/baichuan-inc/Baichuan-13B-Chat/d8e1124426fb781d50266f22be116243b093774d/modeling_baichuan.py", line 527, in stream_generator for token in self.generate(input_ids, generation_config=stream_config): File "/home/kemove/anaconda3/envs/instructor/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 35, in generator_context response = gen.send(None) File "/home/kemove/anaconda3/envs/instructor/lib/python3.10/site-packages/transformers_stream_generator/main.py", line 931, in sample_stream outputs = self( File "/home/kemove/anaconda3/envs/instructor/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/home/kemove/anaconda3/envs/instructor/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward output = old_forward(*args, **kwargs) File "/home/kemove/.cache/huggingface/modules/transformers_modules/baichuan-inc/Baichuan-13B-Chat/d8e1124426fb781d50266f22be116243b093774d/modeling_baichuan.py", line 382, in forward outputs = self.model( File "/home/kemove/anaconda3/envs/instructor/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/home/kemove/.cache/huggingface/modules/transformers_modules/baichuan-inc/Baichuan-13B-Chat/d8e1124426fb781d50266f22be116243b093774d/modeling_baichuan.py", line 325, in forward layer_outputs = decoder_layer( File "/home/kemove/anaconda3/envs/instructor/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/home/kemove/anaconda3/envs/instructor/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward output = old_forward(*args, **kwargs) File "/home/kemove/.cache/huggingface/modules/transformers_modules/baichuan-inc/Baichuan-13B-Chat/d8e1124426fb781d50266f22be116243b093774d/modeling_baichuan.py", line 175, in forward hidden_states = self.input_layernorm(hidden_states) File "/home/kemove/anaconda3/envs/instructor/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/home/kemove/anaconda3/envs/instructor/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward output = old_forward(*args, **kwargs) File "/home/kemove/.cache/huggingface/modules/transformers_modules/baichuan-inc/Baichuan-13B-Chat/d8e1124426fb781d50266f22be116243b093774d/modeling_baichuan.py", line 62, in forward return self.weight * hidden_states

请问这个怎么处理

Young1993 avatar Jul 12 '23 05:07 Young1993