jan
jan copied to clipboard
Published
20 hours ago •
janhq
janhq
feat: show model name & inference time in LLM's reply
Problem
Users can't see how long a LLM response took in total & what model goes with it to make better model selection decision in the future.
Success Criteria
- Show total inference time after response is complete
- Save inference time log for 7 days or 30 days depends on the difficulty in implementation (app pod to decide)
- If we show inference time history, it makes more sense if it goes with the model name as well, as we allow users to switch model mid-thread.
Design
Figma link: https://www.figma.com/file/ytn1nRZ17FUmJHTlhmZB9f/Jan-App?type=design&node-id=3829-165875&mode=design&t=KwLjqwan1T6NiYfy-4
When hover on users' request:
See actions: Edit, copy, delete
When hover on a full assistant's response:
- See total inference time (response time)
- See token speed
- See associated model
- See actions: Regenerate, copy, delete
Archive Nicole's original comment:
Problem Users can't see how long a llm response took in total
Success Criteria A clear and concise description of what you want to happen.
- Show total inference time after response is complete
- State persistence wise, I think we only need to save most recent? Any objections?
- I believe @imtuyethan already has designs for this
Additional context Add any other context or screenshots about the feature request here.
