llx

我是用多卡推理时也出现了这个问题

Maybe you could try using huggingface/accelerate to load the actual weight to your module [https://github.com/huggingface/accelerate](url) ``` from accelerate.utils import named_module_tensors, set_module_tensor_to_device ... for name, _ in named_module_tensors(module): set_module_tensor_to_device(module, name, your_execution_device)...

Hi, I encountered same problem when building [flashinfer](https://github.com/flashinfer-ai/flashinfer), have you guys figured out how to fix that?

You can find it in Huggingface: [Link](https://huggingface.co/datasets/mit-han-lab/pile-val-backup)，but after i downloaded the dataset and use it, it output that dataset is corrupt and unusable

Hi, you can try extracting gradio's inference operations manually, as in the following code ``` if args.model_type == 'vicuna': chat_state = default_conversation.copy() else: chat_state = conv_llava_llama_2.copy() video_path = "your_path" chat_state.system...