TensorRT-LLM
TensorRT-LLM copied to clipboard
[Feature request] Add LogitsProcessor class support in C++ Executor API
Hi team, I would like to use the LogitsPostProcessor in the C++ Executor API to control the generation of language models. However, unlike frameworks like Hugging Face, vLLM, or the implementation in Model Runner, which support class-based approaches, this feature currently only supports functions. This limitation makes implementation challenging. Could the TensorRT-LLM team consider adding support for this feature in TensorRT-LLM? Thank you.