0xd8b issues

Repositories
Issues
Comments

Results 2 issues of


                                            0xd8b

"High GPU usage leads to NaN values in the encoder output of the T5 model (float16).

### System Info GPU: NVIDIA A100 TensorRT-LLM version 0.9.0.dev2024031900 ### Who can help? @symphonylyh @byshiue ### Information - [X] The official example scripts - [X] My own modified scripts ###...

bug

triaged

neeed more info

[Bug]: frequency_penalty Parameter Not Working in TensorRT-LLM RC1.1.0rc5

### System Info TensorRT-LLM Version: RC1.1.0rc5 Model: Qwen/Qwen3-14B (same issue occurs with other models) Command: trtllm-serve /data/Qwen3-14B/ --port 8000 --host 0.0.0.0 --kv_cache_free_gpu_memory_fraction 0.9 --extra_llm_api_options default_config.yaml default_config.yaml : enable_iter_req_stats: True return_perf_metrics:...

bug

Decoding