Feature request: Ability to provide your own feature mask in ModelRunner class.

[noahnisbet]

trafficstars

https://github.com/NVIDIA/TensorRT-LLM/blob/3d56a445e8ebf888e78be638faf6beec0a78f3c2/tensorrt_llm/runtime/generation.py#L121C1-L129C51

Hi,

I am trying to use a unique feature mask with the ModelRunner class. I am not seeing any functionality to do so. I linked to the logic I am seeing for the creation of attention masks above. Could we add this functionality to the class if it does not exist? I'm happy to help implement it.

Thanks!

[hello-11]

cc @AdamzNV @ncomly-nvidia @laikhtewari for vis.

[poweiw]

@ncomly-nvidia any updates on this?

[karljang]

@noahnisbet, Apologies for the very delayed response. Is this ticket still relevant to you? If so, could you share more details about the feature you are considering?

[github-actions[bot]]

Issue has not received an update in over 14 days. Adding stale label.

[karljang]

Closing issue as stale, please feel free to open new one if the problem persists.