TensorRT-LLM icon indicating copy to clipboard operation
TensorRT-LLM copied to clipboard

[Feature Request] Using TRTLLMEncDecModel from the TensorRT-LLM python package

Open guilherme-pombo opened this issue 1 year ago • 0 comments

Hello, thank you for this great project! Currently when converting a T5 model to TRT format, if I then want to use it from another codebase I need to clone this repository and access the class definition from examples/enc_dec, as

from examples.enc_dec import TRTLLMEncDecModel

this is not ideal for many reasons but above all it makes it harder to use TensorRT-LLM from other projects and causes unecessary attrition. Would it be possible to refactor the code in such a way that after I pip install TensorRT-LLM I can import that class as follows:

from tensorrt_llm.enc_dec import TRTLLMEncDecModel

or something of a similar nature? I reckon this could really help increase TensorRT-LLMs adoption as well. Thank you very much.

guilherme-pombo avatar Feb 14 '24 15:02 guilherme-pombo