Jay Zhou
Results
2
comments of
Jay Zhou
Hi, I see that there is a parameter that can be passed by requests here to return logprobs https://github.com/sgl-project/sglang/blob/d3fc86a43e2287e0446a4b3c9acf1300611f1f85/python/sglang/srt/managers/router/model_rpc.py#L222-L223 Is there a way that we could specify this from the...
Many thanks! Currently I would need logprobs of top-5 (or top-n passed as parameter) tokens for each generated token. The scenario is essentially the same as passing the `top_logprobs=n` parameter...