stash-box icon indicating copy to clipboard operation
stash-box copied to clipboard

[RFC] Improved edit count calculation; current method is flawed

Open echo6ix opened this issue 1 year ago • 0 comments

Problem

Currently, accepted edit counts serve primarily as a metric for gauging a user's trustworthiness and participation. However, in the context of proposed leaderboards and/or gamification (see #417, #569), the existing method of calculating edits should be recalibrated to increase accuracy and mitigate manipulation.

Accuracy

The current method of calculating accepted edits falls short of accurately reflecting a user's contributions. A submission where only one field changed is equal to a submission where where the maximum amount of possible fields has changed. That is, all successful submissions are equal to 1 edit point, regardless of how many fields within that submission were effected.

Manipulation

The current method of calculating accepted edits makes it easy to game the edit count. A user can simply bloat their total edit count by submitting multiple small edits for any given entity instead of consolidating their edits into one submission for said entity.

Proposal

Recalibrate how we calculate successful edits. Specifically, each field successfully edited/changed should count as an edit. Using this approach, users will be rewarded edit points based on the granularity of their submissions rather than the number of submissions made.

A submission where only one field is edited = 1 edit A submission where 20 fields are edited = 20 edits

Benefits

  • Fairness: The algorithm ensures a fairer representation of a user's contributions by considering the extent of their edits within each submission.

  • Mitigation of Purposely Inflated Edit Counts: By tying the edit count to the number of fields edited, users will not be able to artificially inflate their edit counts by splitting edits into smaller, less substantial changes.

Other Considerations

Failed/Rejected Edits

How would we calculate rejected or failed edits? An edit will fail for many reasons, including if only one field is undesirable in an otherwise sound submission containing multiple good edited fields. Until a better approach is conceived we might want to keep the failed/rejection count on a per submission basis rather than a per field level.

echo6ix avatar Jan 04 '24 04:01 echo6ix