glicko2js
glicko2js copied to clipboard
mmai
[Suggestion] Change affect of races on rating, deviation, and volatility
To my knowledge Dr. Glickman doesn't talk about >2 person competitions at all, so the math behind the Race features is unique to this module. My understanding is that the way it's currently implemented, when people compete in a race the rating system treats that as if they each participated in a separate complete match with each other player.
This implies that if a player loses a game with 7 other competitors, it would be the same as losing to each of those competitors individually. While I would agree that losing a game with 7 competitors should be more impactful than losing 1 game to 1 competitor, I don't think it's the same as losing 7 separate games to single competitors. It ignores volatility and applies way too much confidence for a single performance.
I don't have a great solution for this. It seems like it would take a modification to the Glicko-2 algorithm itself to make this accurate. It "feels" like it should be akin to losing 7 games for how it affects your score, but should only affect your volatility as a single game would. Confidence would maybe be somewhere in the middle? I'm not sure.
If someone who has a better grasp of the algorithm itself and how all the pieces fits together has an idea of how this should work, it would be great if you chimed in.
What would be helpful in the interim would be some kind of dynamic weighting that could be passed into a match that would reduce it's overall affect on the results. Either something global, like "matchWeight" or something more granular, like "ratingWeight", "rdWeight", and "volWeight". These could be globally configured before every match, or passed into the Race and match objects.
If none of that, I'll probably need to do some kind of post processing hack, or download and edit the source rather than using the package, neither of which are great.
@andimus Yeah I've always felt like the Race feature is a bit out of scope of this module. In my opinion it is better off as a separate module for those who explicitly need it.
Here's a recent talk that Dr. Glickman gave on rating multi-competitor games: A stochastic rank ordered logit model for rating multi-competitor games and sports (video) based on this paper.
That's very helpful, although it does not feel like something I'll be able to implement quickly enough to be worth it. Any chance you're aware of an existing modules for this-- or have some advice on tweaking Glicko-2 to get a similar result?
@andimus I am not aware of any existing implementations. The math in the paper is admittedly out of my league so I probably wouldn't write such a module either :P
You are right, the race feature is is kind of a workaround, I will update the Readme to make this more clear. Thanks for the links @KenanY, I will take a look at the paper, but I can't promise a quick implementation either.
To solve this I implemented it in this way:
Player 1 lost match with 20 kills Player 2 won match with 3 kills Player 3 lost match with 2 kills Player 4 won match with 30 kills
Player 1 vs Player 2
Player 1 gets a tie because he lost the match, but beat Player 2 head to head Player 2 gets a tie because he won the match but lost head to head
Player 1 vs Player 3
Player 3 gets a loss because he lost the match and lost head to head Player 1 gets a tie because he lost the match, but won head to head
Player 4 vs Player 1
Player 4 gets a win because he won the match and won head to head Player 1 gets a loss because he lost the match and head to head
This seems to kind of even things out a bit when it comes to multi-user matches
Umm... what about this: http://www.tckerrigan.com/Misc/Multiplayer_Elo/
This suggest to think of each player as having played two matches: a loss vs. the player right above him on the list, and a win vs. the player right below him.
