chatboxai
[BUG] Auto-generated titles are too terse and vague
Bug Description The automatically-generated titles are too short and vague.
Expected Results The prompt should be adjusted so the AI creates a title that clearly summarizes the topic and distinguishes the conversation from others.
Screenshots
Desktop (please complete the following information):
Web version or desktop, isn't relevant.
I agree. It would also be great if we could choose a different model to generate those titles. GPT-3.5 or Haiku are good enough to create titles, and that would save users a little money.
@Barafu Do you know what model it uses? Agreed it should use a cheap model for the title
@endolith It uses the same model that was used for chat. It is very visible in the logs of token usage, since it reads ~500 tokens and outputs ~10.
It also looks like it puts the entire conversation in the system message instead of the "generate a title" prompt in the system message and the conversation in a user message? Is that why it needs to duplicate the instructions?
import { Message } from '../stores/types'
export function nameConversation(msgs: Message[]): Message[] {
const format = (msgs: string[]) => msgs.map((msg) => msg).join('\n\n---------\n\n')
return [
{
id: '1',
role: 'system',
content: `Name the conversation based on the chat records.
Please provide a concise name, within 10 characters and without quotation marks.
Please use the speak language in the conversation.
You only need to answer with the name.
The following is the conversation:
\`\`\`
${format(msgs.map((msg) => msg.content))}
\`\`\`
Please provide a concise name, within 10 characters and without quotation marks.
Please use the speak language in the conversation.
You only need to answer with the name.
The conversation is named:`,
},
]
}
and what format is the msg.content? If it contains code sections with three backticks, then the prompt needs to be one level higher and have 4 backticks.
@endolith Just for reference: There is a web based chat UI that I used before, and it creates the most useful titles for chats of all I tested. I asked the developer, and he shared the prompt template.
Generate a title in less than 6 words for the following message (language: ${lang}):\n"""\nUser: ${user_message}\nAssistant: ${assistant_message}\n"""
As usual with LLMs, less is more. Note the absence of the system prompt ("You are a helpful ~~assasin~~ assistant bla bla")
@Barafu Well I tried to improve the prompt. I did two tests of it with gpt-3.5 and it worked better than the current prompt. If you want, you can try both prompts and see which produce better titles and I can change https://github.com/Bin-Huang/chatbox/pull/1397 if yours works better
Please allow the user to modify this prompt. It makes sense in Chinese, where 10 字 can express a concept, but in English, it results in strange titles.
The left panel of Chatbox is not resizable, which means that a long title can be unreadable. If it was resizable, it would solve this problem. Typically, a title that is too long is elided with …, and the full title is shown if the panel is expanded, or as a tooltip when the user hovers over it. Even if the left panel remains non-resizable, it is still able to hold more than 10 characters. Perhaps 20.
