TensorRT-LLM icon indicating copy to clipboard operation
TensorRT-LLM copied to clipboard

Fix prompt_table_data empty tensor shape error

Open BasicCoder opened this issue 1 year ago • 2 comments
trafficstars

When executing LLM in a VLM model alone, the correct prompt_table_data's hidden_size = self.hidden_size * tp_size, because self.hidden_size = pretrained_config.hidden_size // tp_size

BasicCoder avatar Nov 20 '24 07:11 BasicCoder

@BasicCoder Thank you for your interest in TrtLLm. We have fixed it in our internal version, and it will be updated on GitHub soon.

hello-11 avatar Nov 26 '24 09:11 hello-11

@BasicCoder Thank you for your interest in TrtLLm. We have fixed it in our internal version, and it will be updated on GitHub soon.

@hello-11 Can you help confirm whether the fix has already landed into the main branch? Then we can decide to close this PR or not.

June

juney-nvidia avatar Mar 28 '25 10:03 juney-nvidia

@BasicCoder Thanks for this contribution on prompt_table_data empty tensor handling!

After reviewing the current codebase (269 days since this PR was created), it appears that comprehensive empty tensor handling has been implemented in the current codebase.

The issue this PR aimed to fix appears to have been addressed through other development efforts. I'm closing this as the functionality is now covered in the current implementation.

Thanks for identifying this issue and contributing to the project!

karljang avatar Aug 19 '25 21:08 karljang