1096125073

batch inference is different with single

9

### System Info x85-64 4 A10 0.9.0 ### Who can help? _No response_ ### Information - [X] The official example scripts - [ ] My own modified scripts ### Tasks...

The engine generated by each build has different results for the same input.

4

### System Info trt-llm v0.9.0 ### Who can help? @byshiue ### Information - [X] The official example scripts - [ ] My own modified scripts ### Tasks - [X] An...

add support for telechat2

2

Two minor changes: 1. Add support for the Telechat model (VLLM now supports the Telechat model). 2. When executing the forward function, it is necessary to place the tensor in...

[Misc]add modules_to_not_convert attribute to gptq series

3

ADD MODULES_TO_NOT_CONVERT ATTRIBUTE TO GPTQ SERIES when using GPTQ, it may be necessary to exclude specific layers, which is a very useful feature.