jan
jan copied to clipboard
bug: Tokens per second calculation is wrong.
Describe the bug Tokens per second is currently calculated including the latency since the beginning of the API request and or hitting the start button.
However, tokens per second should be calculated like this
(Total tokens) / (Time to last token - Time to first token)
Steps to reproduce Steps to reproduce the behavior:
Use jan.ai and observe the token per second counting behaviour is wrong
Expected behavior (Total tokens) / (Time to last token - Time to first token)
Screenshots N/a
Environment details
- Operating System: Windows 11
Logs If the cause of the error is not clear, kindly provide your usage logs: https://jan.ai/docs/troubleshooting#how-to-get-error-logs
Additional context Add any other context or information that could be helpful in diagnosing the problem.