Anindyadeep comments

Results 88 comments of


                                            Anindyadeep

Official Triton Inference Server Image with TRT-LLM support has no TRT-LLM

I see, got it, thanks, will do

Smarter selection of Batch size during hyper-parameter selection and training

Yes, as we are more into automated hyper-parameter selection, so the batch size should also need to be automated. However, I am keeping that to Todo and have kept a...

Step wise resume of prompt2model CLI

Okay, so is it possible to assign this to me then it will be inside the todo

Incorrect results reported for TensorRT-LLM

Hey, sorry for the late reply, however, upon seeing that, I did cross check, and it is to be noted that the device I am using is an A100 80...

Incorrect results reported for TensorRT-LLM

> I expect that A100 would have better absolute performance, but similar relative performance -- when generation is short, the token/s as measured by current code is off the charts...

Incorrect results reported for TensorRT-LLM

Thanks @lopuhin, will update you very soon

Use exllamav2's smart 4-bit KV cache for memory benchmark

We can even add ExLlamav2 for float16 based on [this](https://www.reddit.com/r/LocalLLaMA/comments/1cr5it9/comment/l3xeujs/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button) comment. This also needs to be checked out.

Anindyadeep

Official Triton Inference Server Image with TRT-LLM support has no TRT-LLM

Smarter selection of Batch size during hyper-parameter selection and training

Step wise resume of prompt2model CLI

Incorrect results reported for TensorRT-LLM

Incorrect results reported for TensorRT-LLM

Incorrect results reported for TensorRT-LLM

Use exllamav2's smart 4-bit KV cache for memory benchmark

Fix dsn_or_db_path in readme

Fix dsn_or_db_path in readme

Fix dsn_or_db_path in readme