Self-help support chatbot that uses documentation

[gabek]

Share your bug report, feature request, or comment.

While in general I'm not very bullish on AI, there is one thing I'd like to try.

I'd like to try training a chatbot model on the Owncast documentation, so people could ask it questions in plain English.

It would help with people having to ask the same questions because they don't read the documentation, and us telling them to read the documentation. In this case, they'd just be given the answer. And because it only knows Owncast-specific documentation, the answer should be consistently accurate.

So the steps are probably

• Find a provider that can host and execute against a custom model.
• Find a way to read and/or scrape our documentation into a format that can be used to train a model.
• Train the custom model with our documentation.
• Build/find a chat interface to send questions to the provider, and get answers back that can be displayed. Vercel has an entire SDK around building chatbots with model providers

While I know less than nothing about how generative large language models work, I did try a quick spike to try out some of these todos.

• Signed up for Cohere. They support hosting models and training custom models. I can't try making a custom model until I sign up for a real plan. But it seems to have what we'd need.
• I threw together a proof of concept chat UI based on Vercel's AI SDK. It works. Next step is to try creating a custom model and pointing the UI at that instead.
• Deploy the UI somewhere.
• Integrate the chat service into our web site.

[gabek]

Ok that's kind of cool.

[aaronpk]

You should make a GitHub bot that responds to all new issues with the AI response 😅

[rmens]

Ok that's kind of cool.

I would make an override for this one. It should just say “No. If you need to ask you don’t know what you’re doing and should stay the hell away from it” 😜

[gabek]

You should make a GitHub bot that responds to all new issues with the AI response 😅

I'm not sure if you're serious, but it's a good idea!

[gabek]

I just got off a call with Kapa.AI who specializes in answering questions based on developer documentation, exactly what we're looking for, and worked better than what I hacked together :). It's even able to index GitHub issues if we'd like (by tag or other metadata). I mentioned we might want to build a Rocket.Chat integration so people who want to drop by our chat can use this chat interface as well, and he said it has an API we can use. So it all sounds pretty great.

He offered a generous free open-source project plan of a couple thousand questions a month as well, so that's even better. I think this is a much better solution.

He's going to get me login credentials at the end of the week, and I'll give it a spin.

[alyssais]

Could I request that the "Ask AI" option not be the default when searching the documentation? I found it very confusing that the documentation search form was not actually searching the documentation, once I even realised what was going on.

[gabek]

It was searching the same documentation, but I know what you mean. And it shall be changed. Good feedback, thanks!

[gabek]

So this has been running for a while. And this is the state of things:

1. There's currently no way to access it on the web. I've reverted search to be search, so you can't access it from there. And I don't know how else you would access it. I don't want to have one of those stupid pages with a floating button in the corner that says "OMG AI ANSWER ALL UR QS".
2. It works well in our chat, but nobody knows about it. I have to manually fire it for people. We have a bot that explains it when you join, but of course nobody reads it.

So currently it hasn't been helpful at all because I don't know how to get it in front of people so they can actually use it.

[gabek]

I suppose if I don't come up with something better, I can add the stupid floating button on the web page.

[edmael]

Sadly the floating button seems to be the universal way of implementing this. Not because it's necessarily the best idea, but mainly because everyone is doing it so people are used to that behavior.

You can always start with the stupid floating button and implement something better later 😉

[gabek]

That's a good point! Thanks for the feedback, @edmael!

[gabek]

After moving to the floating button, the number of questions that were answered for people certainly jumped up. So I guess that is, indeed, the best answer for now!