Ma Junteng

> Also getting wrong self.world_size when using zero3 while zero2 is alright in the same environment (8xA100) > > i have deepspeed==0.11.1, transformers==4.34.1, accelerate==0.24.0 and ray==2.7.1 hi, did u solve...

> 感谢提醒，找个时间看看 您好，请问这个转换问题解决了嘛~

相似的问题，我是模型下载到本地之后，使用AutoModelForCausalLM来加载： `model = AutoModelForCausalLM.from_pretrained( model_path, trust_remote_code=True, device_map="auto", torch_dtype=torch.bfloat16, )` model_path是我本地的目录，提示：ModuleNotFoundError: No module named 'transformers_modules.openbuddy-falcon-7b-v1。 请问在模型文件在本地时应该如何加载呢？

> > 相似的问题，我是模型下载到本地之后，使用AutoModelForCausalLM来加载： `model = AutoModelForCausalLM.from_pretrained( model_path, trust_remote_code=True, device_map="auto", torch_dtype=torch.bfloat16, )` model_path是我本地的目录，提示：ModuleNotFoundError: No module named 'transformers_modules.openbuddy-falcon-7b-v1。 请问在模型文件在本地时应该如何加载呢？ > > 好像mode_patch最后要加个`/`，我再确认一下 您是对的，需要在路径后面加个斜杠，谢谢所提供的模型，我再去体验下，希望加载路径的方式能够统一一下~

> Here https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/llama#run-llama-with-several-lora-checkpoints is example of multi-lora. Hoping it is helpful. hi, i am very instrested in multi-lora method! What should i do if i need the base model to...

> TRT-LLM does not support quantization on lora case now. So in multi-lora scenario, the base model and lora adpter should both fp16/bf16?