DeepSpeed-VisualChat Tensor shape mismatch

[Linjiahua]

When I use the CLI interface I get some errors, is this a bug?

Enter image pathes, seperate by space (only support one image per time for now) (type 'na' for empty image): /root/ljh/lab/DeepSpeedExamples/applications/DeepSpeed-VisualChat/chat/bridie.jpg Enter input (type 'quit' to exit, 'clear' to clean memory): Traceback (most recent call last): File "/root/ljh/lab/DeepSpeedExamples/applications/DeepSpeed-VisualChat/chat/chat.py", line 169, in main() File "/root/ljh/lab/DeepSpeedExamples/applications/DeepSpeed-VisualChat/chat/chat.py", line 155, in main generate_output = model.generate(image_tensor, input_ids, generation_length=args.generation_length_per_round) File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/root/ljh/lab/DeepSpeedExamples/applications/DeepSpeed-VisualChat/utils/model/modeling_dsvl.py", line 366, in generate output = self.lang_decoder.generate(input_ids=None, File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py", line 1648, in generate return self.sample( File "/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py", line 2730, in sample outputs = self( File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1505, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1514, in _call_impl return forward_call(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/accelerate/hooks.py", line 164, in new_forward output = module._old_forward(*args, **kwargs) File "/root/ljh/lab/DeepSpeedExamples/applications/DeepSpeed-VisualChat/utils/model/third_party_model/hf_model/modeling_llama.py", line 907, in forward logits = self.lm_head(hidden_states) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1505, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1514, in _call_impl return forward_call(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/accelerate/hooks.py", line 159, in new_forward args, kwargs = module._hf_hook.pre_forward(module, *args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/accelerate/hooks.py", line 286, in pre_forward set_module_tensor_to_device( File "/usr/local/lib/python3.10/dist-packages/accelerate/utils/modeling.py", line 285, in set_module_tensor_to_device raise ValueError( ValueError: Trying to set a tensor of shape torch.Size([32000, 8192]) in "weight" (which has shape torch.Size([32003, 8192])), this look incorrect.