Ankit Mathur

Perhaps the problem is that this script does not generate the Triton config? https://github.com/fauxpilot/fauxpilot/blob/main/converter/download_and_convert_model.sh

cc: @fdegier @thakkarparth007 I see you guys just committed to this file - do you know what might be going on here? The triton config generator is not even included...

@thakkarparth007 no worries! yes, definitely would love to see it in setup.sh! I've been hacking some stuff together to try to get that file generated for a 4GPU setup, but...

@thakkarparth007 @MichaMucha thanks for your help! I did get this working by just using that script and then using the rebase argument in the script to use `/model` instead of...

Yup! Haven't had time to put up a PR yet unfortunately

Hi folks! Thanks so much for surfacing the issue, and apologies about the delay in responding. @daanknoope , thanks for the descriptive analysis of the bug - I agree with...

Quick ping here @daanknoope

Thanks @daanknoope - please add me as a reviewer and I'm happy to take a look when you have a contribution ready! Agree - we could add a note to...

https://github.com/NVIDIA/FasterTransformer/issues/211#issuecomment-1093495810

@hujiaxin0 what precision problem are you referring to?