Zhao Shenyang

to solve: - [ ] auto DataContainer recognize multiple outputs

feature(frameworks): bentoml.onnx runner accept kwargs

5

## What does this PR address? Sometimes it's more nature to call onnx model with keyword arguments. For example, bert's tokenizer will output a dictionary, which the original model will...

## What does this PR address? Fixes #(issue) ## Before submitting: - [ ] Does the Pull Request follow [Conventional Commits specification](https://www.conventionalcommits.org/en/v1.0.0/#summary) naming? Here are [GitHub's guide](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request) on how to...

InFlightBatching seems not working

3

### System Info - CPU: amd64 - OS: Debian 12 - GPU: nvidia rtx4000 ada - GPU driver: 535.161 - TensorRT-LLM version: 0.8 - tensorrtllm_backend version: 0.8 ### Who can...

[Bug]: Pixtral inference not working correctly with LLMEngine/AsyncEngine

3

### Your current environment The output of `python collect_env.py` ```text Collecting environment information... PyTorch version: 2.4.0+cu121 Is debug build: False CUDA used to build PyTorch: 12.1 ROCM used to build...