online-judge
online-judge copied to clipboard
DMOJ
Community Voted PP
A frequent complaint among DMOJ users is that problems do not have the correct pp rating. There's no clear way to avoid this issue at all because difficulty is subjective, but one way to help resolve obvious issues with pp is to allow the community to give their opinions explicitly on pp for a problem. This has the added benefit of giving other users a frame of reference for the difficulty of the problem.
A sample design goes as follows:
- In order to be eligible to vote, you must have already earned at least some number of pp on the platform. This should be set fairly high since it takes a decent amount of experience to really understand problem difficulty. This has the positive side effect of reducing the number of trolls who can pollute the rating.
- There should be a way to ban users from voting if they do decide to exploit this privilege. All of their votes would subsequently be removed.
- A user is allowed to submit their pp vote for a problem. Either all explicit opinions can be shown to all users, or we can surface just some metadata (number of votes, mean/median vote, etc) publicly, and give access to the full set of votes in the admin panel.
This would not go the extra step into somehow automatically incorporating the community votes into updating the pp of the problem dynamically. However, it would now be possible to see which problems are the most egregiously in violation of not being at the proper pp.
There is some active development happening on this front. Here are some things that were discussed that are slightly different from the original proposal:
- A necessary and (generally sufficient) condition to vote on a problem is solving it. If we do decide to use the votes in some automated fashion, all votes should be considered equally.
- Unlisted users should not be allowed to vote. When a user is unlisted, all of their votes should be retroactively removed.
- Users will vote by inputting an integer number of pp between 1 and 50.
