transformer-deploy icon indicating copy to clipboard operation
transformer-deploy copied to clipboard

Published 20 hours ago • verified publisher icon ELS-RD

Llama support

Open michaelroyzen opened this issue 1 year ago • 2 comments

Would it be possible to run llama using this? Is the gpt2 example hackable to run llama on tensorrt?

michaelroyzen avatar Apr 19 '23 23:04 michaelroyzen

My question also^ - very much interested in trying to get LLMs like Llama to work on Triton CC @pommedeterresautee

ktl014 avatar May 08 '23 14:05 ktl014

Also very interested

tikikun avatar Jul 08 '23 22:07 tikikun