jan icon indicating copy to clipboard operation
jan copied to clipboard

bug: Tokens per second calculation is wrong.

Open avianion opened this issue 1 month ago • 3 comments

Describe the bug Tokens per second is currently calculated including the latency since the beginning of the API request and or hitting the start button.

However, tokens per second should be calculated like this

(Total tokens) / (Time to last token - Time to first token)

Steps to reproduce Steps to reproduce the behavior:

Use jan.ai and observe the token per second counting behaviour is wrong

Expected behavior (Total tokens) / (Time to last token - Time to first token)

Screenshots N/a

Environment details

  • Operating System: Windows 11

Logs If the cause of the error is not clear, kindly provide your usage logs: https://jan.ai/docs/troubleshooting#how-to-get-error-logs

Additional context Add any other context or information that could be helpful in diagnosing the problem.

avianion avatar May 18 '24 00:05 avianion