Julien Demouth comments

Results 7 comments of


                                            Julien Demouth

Nvidia Jetson device Support

Hi @shahizat , We do not officially support Orin yet but we have colleagues (in our automotive division) who are working on enabling TensorRT-LLM on Orin. I’ll ask them if...

pip install -e . does not work

@Shixiaowei02 , can you help with that issue, please?

pip install -e . does not work

@wangkuiyi , for your information, @Shixiaowei02 is based in China. It means that he won't be able to work on this issue before the end of the break for the...

Any support for RWKV plz?

Hi @Pevernow , thanks for your message. For the moment, RWKV is not on our roadmap. However, we welcome external contributions and if you are willing to contribute an implementation...

Under the main branch, stress testing the in-flight Triton Server with multiple threads can result in the Triton Server getting stuck.

@StarrickLiu , we have been trying to reproduce the hang for a few days and we have not been able to do it so far. Have you been able to...

Llama-2-7b - Tensor Dimension Mismatch Error for AWQ Engine Build for 4 GPUs

Can you test if the issue persists in the `main` branch, please? I see fixes for Llama AWQ TP > 1 in our internal repo that were pushed to the...

Illegal memory access when medium batch sizes on using bad_words

@0xymoro can you verify if the issue persists with Llama-v2-7B and TP=2 or TP=4, please? Also, are you using the `main` branch?