Create `/top-helper-ai` slash command

[surajkumar]

Description

This PR introduces a new slash command /top-helper-ai. The idea of this command is to use ChatGPT to determine who the top helpers of the current month are.

The command works like this:

1. Loop over all the questions in the #questions forum channel
2. Break if the question was not created since the start of the current month
3. Fetch all the messages that were sent in a given question
4. Send message to ChatGPT asking who the helper was in that given question
5. Fetch the users that appeared as the "top helper"
6. Respond with an embed showing the results

This PR is the first take. In a thread that has multiple top helpers, only 1 of them will be the "top" helper. We'll see how it goes. Should this work very well, the next step would be to enchance it to detect multiple helpers in a given question.

Why even create this command?

The TJ moderators currently determine the top helper based on message count. The more messages you have posted in the questions forum, the more likely you are to be the top helper. This includes "shit posting" and if you're a recognised member (say, you're fairly active), the team might confuse your shit post with actual helpful content.

Top helper is a role given to people that provide quality help in the questions forum. As such, we often reward helpers with Intellij licenses and other incentives so it's pretty crucial that we are awarding such things to users that actually post helpful responses to questions.

Using ChatGPT is an attempt to better this process to keep the questions forum a helpful space.

The prompt sent to ChatGPT

The prompt sent to ChatGPT looks like this:

The following contains user IDs and their message. Using the messages provided by each user, which user ID was the most helpful/answered the question. If there are no meaningful messages, you must still choose somebody. ONLY provide the user ID of that person. Do not reply with anything else.
%s

After that prompt, the %s is replaced with all the messages in the following syntax; <userId>: <message> A sample:

123456789: Try doing xyz
334456789: it didn't work
123456789: ok try this now, zyx

In this example, we would hope that 123456789 is the top helper.

Code exludes the OP and bots from being the top helper.

Full prompt sent:

The following contains user IDs and their message. Using the messages provided by each user, which user ID was the most helpful/answered the question. If there are no meaningful messages, you must still choose somebody. ONLY provide the user ID of that person. Do not reply with anything else.
123456789: Try doing xyz
334456789: it didn't work
123456789: ok try this now, zyx

Command output:

Embed containing:

Top helpers

user1 10
user2 5
user3 3
user4 1

The higher the number next to their name, the better the helper they are

[Zabuzard]

btw "The TJ moderators currently determine the top helper based on message count" is not true. the current command is based on message length, not count.

i havent seen anyone yet who attempted to influence it with ill intention. no one was exploiting it yet. the only bias it really has is favoring posts with code, as they obviously tend to have a lot more characters than pure text answers.

i would be surprised though if this actually yields results really different to the ones we already have, so far the top helper lists always looked very promising.

but that just as a note, im not against the idea of using chatgpt to help. if anything, then my comments would probably rather be at how this particular implementation is using discord api to fetch a massive load of content. instead of us just saving stuff in a database or sth and fetching from there.

sth that is also interesting to check out is how this would influence our chatgpt api cost, as the amount of content send by this is quite massive. a rough estimate on the cost would be nice. but given that we are currently paying almost nothing, there is still a lot of room for using chatgpt more.

[surajkumar]

Thanks @Zabuzard, around message count, that was an accident. You're right it's message length.

I'm currently evaluating the costs so I'll respond back once I can give us a rough idea around the assumption we always get mega threads so we can understand our worst case scenario.