feat: Thread titles should auto-summarize Topic

[dan-menlo]

Our current Jan results in "New Thread" for each thread, as users don't title each thread manually.

We should find a way to implement a ChatGPT-style Thread Title summarizer, using the underlying LLM (prompt?)

[hahuyhoang411]

I thought about this before. But don't know how it should work. Maybe we can pre-download Tinyllama for users, and then we can use it as a title maker?

[hiro-v]

We can use the existing model for summarizing threads message after the first question/ answer pair and fill out the thread. Do not need to use another model.

[0xgokuz]

Hi @dan-jan, I'm taking on this task

[hantran-co]

@0xgokuz Hi, can I check how's the progress of this task?

[0xgokuz]

@0xgokuz Hi, can I check how's the progress of this task?

Hi @imtuyethan - I'm going to clean up the solutions, and going through it one more time before requesting another PR. Hopefully I can get that sorted this week / next latest.

[Van-QA]

hi @0xgokuz, thank you for your contribution,

While testing the generated thread title, occasionally the title is too long, which can cause inconvenience for users, could we establish a restriction on the thread titles generated to guarantee they are brief and user-friendly? (e.g. around 10 words, etc...)

cc: @louis-jan

[0xgokuz]

@Van-QA it's very dependent on the model that you're using actually. I noticed that the title is significantly cleaner if you're using GPT4. Try it out.

[Van-QA]

Tested and looking good on Jan v0.4.6-273 ✅