MLServer icon indicating copy to clipboard operation
MLServer copied to clipboard

Published 20 hours ago • verified publisher icon SeldonIO

Support token metrics

Open luohua13 opened this issue 11 months ago • 1 comments

VLLM runtime has a wealth of token metrics, example prompt_tokens_total and generation_tokens_total. Why does mlserver have none?

luohua13 avatar Dec 24 '24 06:12 luohua13