Open-Assistant icon indicating copy to clipboard operation
Open-Assistant copied to clipboard

XP thresholds for level progression

Open sedthh opened this issue 2 years ago • 1 comments

I have finalized the xp thresholds discussed in https://github.com/LAION-AI/Open-Assistant/issues/996 for gamification purposes.

The forumla for the linear progression curve used for calculating the levels is:

def get_thresholds(baseline: int = 3, alpha: float = 1.1521, max_level: int = 100) -> np.ndarray:
    level = np.round(np.cumsum(np.arange(1, max_level) * alpha + baseline))
    return np.array([0] + level.astype(int).tolist())

which yields the following level: xp needed thresholds:

1  :    0	2  :    4	3  :    9	4  :   16	5  :   24
6  :   32	7  :   42	8  :   53	9  :   65	10 :   79
11 :   93	12 :  109	13 :  126	14 :  144	15 :  163
16 :  183	17 :  205	18 :  227	19 :  251	20 :  276
21 :  302	22 :  329	23 :  357	24 :  387	25 :  418
26 :  449	27 :  482	28 :  516	29 :  552	30 :  588
31 :  626	32 :  664	33 :  704	34 :  745	35 :  787
36 :  831	37 :  875	38 :  921	39 :  968	40 : 1016
41 : 1065	42 : 1115	43 : 1166	44 : 1219	45 : 1273
46 : 1327	47 : 1383	48 : 1441	49 : 1499	50 : 1558
51 : 1619	52 : 1681	53 : 1744	54 : 1808	55 : 1873
56 : 1939	57 : 2007	58 : 2075	59 : 2145	60 : 2216
61 : 2288	62 : 2362	63 : 2436	64 : 2512	65 : 2588
66 : 2666	67 : 2745	68 : 2825	69 : 2907	70 : 2989
71 : 3073	72 : 3158	73 : 3244	74 : 3331	75 : 3419
76 : 3508	77 : 3599	78 : 3691	79 : 3784	80 : 3878
81 : 3973	82 : 4069	83 : 4167	84 : 4265	85 : 4365
86 : 4466	87 : 4568	88 : 4671	89 : 4776	90 : 4881
91 : 4988	92 : 5096	93 : 5205	94 : 5315	95 : 5426
96 : 5539	97 : 5652	98 : 5767	99 : 5883	100: 6000

This allows fast initial progression for newcomers, while only the top 12 players or so will be capped at 100 as of now. Later on we can extend the maximum number of levels, but the top players will always be outdoing the curve, no matter what. I think it's fine, as they are driven by other reasons anyway.

Calculate the user's current level via:

thresholds = get_thresholds()
current_ponts = UserStats.compute_leader_score()

current_level = (thresholds <= current_points).sum()   # lowest level is 1

and to generate a 0.-1. progress bar:

if current_level >= len(thresholds)
    prograss = 1.
else: 
    progress = (current_points - thresholds[current_level - 1]) / (thresholds[current_level] - thresholds[current_level - 1])

In case it's fine to just put these funcitons in UsrStats https://github.com/LAION-AI/Open-Assistant/blob/main/backend/oasst_backend/models/user_stats.py then I can do a PR, but if it's more complicated than that, then I would like to ask someone from the website / backend team to implement it.

NOTE: we should let people know that adding only prompts means less points, because otherwise it would take to write 35 (unranked) prompts for a newcomer to reach level 2, instead of simply writing an answer.

sedthh avatar Mar 04 '23 12:03 sedthh

Currently the website asks the backend via endpoints api/v1/users/{id}/stats/{timeframe} or api/v1/leaderboards/{timeframe} ... and uses the UserScore/LeaderboardStats model class defined in protocol.py The computation of the XP-level could for example be added to user_stats_repository.py ... _create_user_score() https://github.com/LAION-AI/Open-Assistant/blob/f8a69ca607acaf1b1e64af6d54fc65346e93076b/backend/oasst_backend/user_stats_repository.py#L41 ..

andreaskoepf avatar Mar 04 '23 17:03 andreaskoepf