Yun Zhao comments

Repositories
Issues
Comments

Results 2 comments of


                                            Yun Zhao

How to test the benchmark of Llama3 and Vicuna2 of TensorRT-LLM by benchmark.py

> Hi @Ourspolaire1 , the most suggested way currently is use `trtllm-build` command to build the models you want to benchmark, and use `gptManagerBenchmark` to benchmark it, please see the...

How to test the benchmark of Llama3 and Vicuna2 of TensorRT-LLM by benchmark.py

@raoofnaushad I met a new error. When I test the benchmark.py with batch_size=1 it works, but when I change the batch_size=2and other numbers the error occurs. Do you know how...