xieziyi881

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:7 and cuda:0! (when checking argument for argument index in method wrapper_CUDA__index_select)

3

`from awq import AutoAWQForCausalLM from awq.utils.utils import get_best_device from transformers import AutoTokenizer, TextStreamer quant_path = "/workspace/awq_model" if get_best_device() == "cpu": model = AutoAWQForCausalLM.from_quantized(quant_path, use_qbits=True, fuse_layers=False) else: model = AutoAWQForCausalLM.from_quantized(quant_path, fuse_layers=True,device_map="balanced")...

I encountered the following problem during the KL assessment

2

ValueError: You are attempting to load an AWQ model with a device_map that contains a CPU or disk device. This is not supported. Please remove the CPU or disk device...

ConnectionError: Couldn't reach 'mit-han-lab/pile-val-backup' on the Hub (ConnectTimeout)

16

I guess it should be a network issue。I'm running inside a docker container,so how to handle it??Thanks ur advise