fastertransformer_backend icon indicating copy to clipboard operation
fastertransformer_backend copied to clipboard

Published 20 hours ago • verified publisher icon triton-inference-server

enable llama model in FT backend

Open shihy52x opened this issue 1 year ago • 1 comments

existing FT backend will throw error for llama model.

shihy52x avatar Jun 24 '23 06:06 shihy52x

Will this ever work? I didn't see llama defined under: https://github.com/NVIDIA/FasterTransformer/tree/main/src/fastertransformer/triton_backend

sfc-gh-zhwang avatar Jul 08 '23 00:07 sfc-gh-zhwang