TensorRT-LLM icon indicating copy to clipboard operation
TensorRT-LLM copied to clipboard
verified publisher icon NVIDIA

How to set `top_p` 'top_k' arguments in `gptManagerBenchmark`?

Open sleepwalker2017 opened this issue 1 year ago • 1 comments

What is the sampling strategy in gptManagerBenchmark?

sleepwalker2017 avatar Apr 22 '24 09:04 sleepwalker2017

gptManagerBenchmark does not support specifying sampling strategy yet, and it's using default top_p and top_k, which is top_p=0.0 and top_k=1.

kaiyux avatar May 22 '24 13:05 kaiyux