TensorRT-LLM
TensorRT-LLM copied to clipboard
Fix prompt_table_data empty tensor shape error
When executing LLM in a VLM model alone, the correct prompt_table_data's hidden_size = self.hidden_size * tp_size, because self.hidden_size = pretrained_config.hidden_size // tp_size
@BasicCoder Thank you for your interest in TrtLLm. We have fixed it in our internal version, and it will be updated on GitHub soon.
@BasicCoder Thank you for your interest in TrtLLm. We have fixed it in our internal version, and it will be updated on GitHub soon.
@hello-11 Can you help confirm whether the fix has already landed into the main branch? Then we can decide to close this PR or not.
June
@BasicCoder Thanks for this contribution on prompt_table_data empty tensor handling!
After reviewing the current codebase (269 days since this PR was created), it appears that comprehensive empty tensor handling has been implemented in the current codebase.
The issue this PR aimed to fix appears to have been addressed through other development efforts. I'm closing this as the functionality is now covered in the current implementation.
Thanks for identifying this issue and contributing to the project!