stash-box
stash-box copied to clipboard
[RFC] Improved edit count calculation; current method is flawed
Problem
Currently, accepted edit counts serve primarily as a metric for gauging a user's trustworthiness and participation. However, in the context of proposed leaderboards and/or gamification (see #417, #569), the existing method of calculating edits should be recalibrated to increase accuracy and mitigate manipulation.
Accuracy
The current method of calculating accepted edits falls short of accurately reflecting a user's contributions. A submission where only one field changed is equal to a submission where where the maximum amount of possible fields has changed. That is, all successful submissions are equal to 1 edit point, regardless of how many fields within that submission were effected.
Manipulation
The current method of calculating accepted edits makes it easy to game the edit count. A user can simply bloat their total edit count by submitting multiple small edits for any given entity instead of consolidating their edits into one submission for said entity.
Proposal
Recalibrate how we calculate successful edits. Specifically, each field successfully edited/changed should count as an edit. Using this approach, users will be rewarded edit points based on the granularity of their submissions rather than the number of submissions made.
A submission where only one field is edited = 1 edit A submission where 20 fields are edited = 20 edits
Benefits
-
Fairness: The algorithm ensures a fairer representation of a user's contributions by considering the extent of their edits within each submission.
-
Mitigation of Purposely Inflated Edit Counts: By tying the edit count to the number of fields edited, users will not be able to artificially inflate their edit counts by splitting edits into smaller, less substantial changes.
Other Considerations
Failed/Rejected Edits
How would we calculate rejected or failed edits? An edit will fail for many reasons, including if only one field is undesirable in an otherwise sound submission containing multiple good edited fields. Until a better approach is conceived we might want to keep the failed/rejection count on a per submission basis rather than a per field level.