Nulooper

icon indicating an email [email protected]

icon location System Architecture, Adaptive Learning, Natural Language Processing

Results 1 comments of Nulooper

TGI performance is better than vllm on A800

I use ab to test, the result is about 97 requests/min on A100 40G 1*gpu.