Open-Assistant icon indicating copy to clipboard operation
Open-Assistant copied to clipboard

Limit the number of responses in ranking task

Open bitplane opened this issue 2 years ago • 2 comments

Currently ranking tasks are pretty difficult when there's too many nodes at the level. Here's an extreme example posted on Discord:

rankreplies

Personally I find it difficult to order even 3 multiple-paragraph responses.

Suggestion: have multiple stages - thumbs up/down the answers, remove thumbs down, if there's more than 3 left then pick the best 3, then order the remaining ones.

bitplane avatar Feb 04 '23 21:02 bitplane

To clarify this, I think we already try to limit the number of replies to 3 or 4 but if users have the task concurrently it can go over. So what we need is a solution which prevents this in future

olliestanley avatar Feb 04 '23 22:02 olliestanley

100ecbe2d53c35c00279b2d1d9aa31754ec265e0 should address this to some extent I believe. If anyone still has problems with this please comment here

olliestanley avatar Feb 04 '23 23:02 olliestanley

We should no longer be seeing huge ranking tasks and I haven't seen any reports of unreasonable amounts recently so this issue seems to be resolved. Feel free to comment again if you are still seeing it.

olliestanley avatar Feb 06 '23 21:02 olliestanley