Implement good rating system
Here is a small update after talking to n_brain:
He was recommending us to take a look for the glicko-2 rating system. There is also a pdf-File with mathematical explanations. n_brain said it would be best in combination with a community-based fairness reward system. Sounds like a really good approach to me, not reinventing the wheel and still practicable for our purpose. I would like to help implementing it in the future.
EDIT: It is also implemented by CS:GO, so the article (saved with waybackmachine) that @chocoladisco linked in his comment is a good read, too. For the actual implementation see the pdf in this comment.
EDIT2: Also the thoughts of the Guild Wars devs are worth a read and also the pvp-matchmaking algortihm explanation from their wiki.
EDIT3: The paper from MS about Trueskill(TM).
EDIT4:
TeamFortress2-Matchmaking algorithm explained. I really like that Competitive Mode approach. Will take some notes from that video and publish them in a gist (link will follow).
EDIT5: A Bayesian Approximation Method for Online Ranking https://web.archive.org/web/20210501042619/https://jmlr.csail.mit.edu/papers/volume12/weng11a/weng11a.pdf
Originally posted by @simonsan in https://github.com/SFTtech/openage-masterserver/pull/9#issuecomment-497490151
There was a new publication by researchers at Microsoft on Trueskill 2: https://www.microsoft.com/en-us/research/publication/trueskill-2-improved-bayesian-skill-rating-system/
And here I provide a link to the original Trueskill that is not on the web.archive: https://proceedings.neurips.cc/paper_files/paper/2006/file/f44ee263952e65b3610b8ba51229d1f9-Paper.pdf
An important fact is that Trueskill is patented.