Lei Sun

Results 10 comments of Lei Sun

All-in-one Meta-Llama-3.1-8B RuntimeError: Expected all tensors to be on the same device, but found at least two devices, xpu:0 and cpu!

I met exact the same problem when run all-in-one benchmark of Llama-3.1 8B.

Ubuntu 22.04 Run Text Generation WebUI on Intel GPU Could not create share link

Have a try "unset http_proxy" before run your python script?

When convert MiniCPM-V model from ModelScope to low-bit, got error: AttributeError: 'NoneType' object has no attribute 'add_bos_token'

pip list |grep transformers shows we are working on transformers 4.37.0, I will try 4.36. Thanks a lot for your quick reply

When convert MiniCPM-V model from ModelScope to low-bit, got error: AttributeError: 'NoneType' object has no attribute 'add_bos_token'

after pip install transformers==4.36.0 or 4.36.2, I got the same error, no change at all.

When convert MiniCPM-V model from ModelScope to low-bit, got error: AttributeError: 'NoneType' object has no attribute 'add_bos_token'

patch the file as follows: posemb = F.interpolate(posemb, size=new_size, mode=interpolation, antialias=antialias) -> posemb = F.interpolate(posemb.to("cpu"), size=new_size, mode=interpolation, antialias=antialias).to(posemb.device) can fix your problem.

When convert MiniCPM-V model from ModelScope to low-bit, got error: AttributeError: 'NoneType' object has no attribute 'add_bos_token'

Would you please help check the version of bigdl-llm? I just tried bigdl-llm==2.4.0, and it does NOT work either. In my env, $ pip list |grep bigdl shows as follows....

When convert MiniCPM-V model from ModelScope to low-bit, got error: AttributeError: 'NoneType' object has no attribute 'add_bos_token'

I checked https://github.com/intel-analytics/ipex-llm/issues/10470, bigdl-llm==2.5.0b20240318, I will have a try.

When convert MiniCPM-V model from ModelScope to low-bit, got error: AttributeError: 'NoneType' object has no attribute 'add_bos_token'

Can I do it with ipex--llm instead of bigdl-llm? Because I find I have NOT installed bigdl-llm.

When convert MiniCPM-V model from ModelScope to low-bit, got error: AttributeError: 'NoneType' object has no attribute 'add_bos_token'

I want to convert the model to int4, Any update?

When convert MiniCPM-V model from ModelScope to low-bit, got error: AttributeError: 'NoneType' object has no attribute 'add_bos_token'

path = "./models/OpenBMB/MiniCPM-V" save_path = "./models/OpenBMB/MiniCPM-V-int4" model = AutoModel.from_pretrained(path, load_in_4bit=True, optimize_model=False, trust_remote_code=True, modules_to_not_convert=["vpm", "resampler"], use_cache=True) model = model.float().to(device='xpu') tokenizer = AutoTokenizer.from_pretrained(path, trust_remote_code=True) model.eval() model.save_low_bit(save_path) tokenizer.save_pretrained(save_path) print(f"Model and tokenizer are saved...