OthersideAI
Generate multiple responses in polling and select the most popular choice? Particularly for -accurate grid overwrite
I was wondering if there was a reason we only picked the top response, or the 0th one. Instead, what if we asked the model to generate 9 responses, and then use the one that popped up the most frequently as the answer?
There's a possibility this wouldn't work for general actions, but I think this would work particularly well for my -accurate grid overwrite where when the model tries to click on something, I simply ask it which grid it would like to click in. Where with 9 responses of a number between 0 - 15, or 0 - 3, I can just use whichever number was most popular.
@klxu03 Curious how much this would affect the token count and cost per API call.
I don't think that much.
Seems like it is input + n * output
If n (the number of completions returned) or best_of (the number of completions generated for consideration) are set to > 1, each request will create multiple outputs. Here, you can consider the number of generated tokens as [ max_tokens * max (n, best_of) ]
https://platform.openai.com/docs/guides/production-best-practices/number-of-completion-tokens
I think most of the API cost will be in embedding the input (tokens, and the image especially)
@klxu03 This could be an interesting approach.
@Sameeraali835 Feel free to join the Discord to discuss any issues you're having. You can also open an issue here on the repository if it could be affecting other users as well.
@klxu03 still believes this approach shows promise. Would love to see a PR for it!
@klxu03 still believes this approach shows promise. Would love to see a PR for it!
ahh yes! was looking into it, looks like you can also just set temperature to like 0 so there's no variability (in the grid choice at least), gpt basically always takes the highest probabilistic token as well
I was having the hardest time with repetitive mouse clicks originally, so I played around with temperature to try to fix it. I should try reducing it and see if repetition is still a problem