Yingqiang Ge

to install the main brach, you can use the following command `pip3 install tensorrt_llm -U --pre --extra-index-url https://pypi.nvidia.com` also check the readme for more detail

I used and it worked, my current version is dev2024020600 your cuda version is too high, follow the instruction and use 12.1 On Thu, Feb 8, 2024 at 00:00 Yi...

When using multiple gpus, i tested mpt-7b and mistral-7b, both failed. And the errors are OOM.

I used this command this morning. `pip3 install tensorrt_llm -U --pre --extra-index-url https://pypi.nvidia.com ` My current version is `[TensorRT-LLM] TensorRT-LLM version: 0.8.0.dev20240123020.8.0.dev2024012302` Do you mean it is still not the...

Thanks, this is my current version `TensorRT-LLM version: 0.8.0.dev2024013000` now. I kept following the instruction of mpt: tp=1 works well, but there are problems when tp=4. Can you re-open this...

#1034 I opened a new issue to discuss, and found someone else also facing similar problem. pls check this out.

model_name = 'cpo' You can check the whole train and test process in FCPO/fcpo/README_CPO-ml-100k.ipynb

Currently not. Just tried to learn the basics of AutoAWQ then met this issue.