Flex Wang

Results 4 comments of


                                            Flex Wang

Python Backend complains "triton_python_backend_utils" has no attribute "InferenceRequest"

why remove type annotation would help? Or maybe I should ask, why use pbu.InferenceRequest would cause issue?

enable llama model in FT backend

Will this ever work? I didn't see `llama` defined under: https://github.com/NVIDIA/FasterTransformer/tree/main/src/fastertransformer/triton_backend

Serve Deberta using FasterTransformer in Triton

https://github.com/NVIDIA/FasterTransformer/pull/725

Add triton fastertransformer backend support for deberta

@byshiue @dwyatte mind taking a look?