Open-Assistant icon indicating copy to clipboard operation
Open-Assistant copied to clipboard

Update user streaks

Open andreaskoepf opened this issue 2 years ago • 3 comments

As part of the gamification layer we want to calculate the number of consecutive days on which users completed human feedback tasks and show them to the users and on the leaderboard.

Currently we have two fields on the UserStats db model:

  • streak_last_day_date
  • streak_days

Todo:

  1. Remove streak_last_day_date & streak_days from UserStats and add them to User (add a server default current_timestamp() for streak_last_day_date) + add a nullable additional last_activity_date datetimetz column to User.
  2. Update last_activity to the current time whenever the user sends a task results, e.g. after tm.handle_interaction() in the tasks_interaction endpoint.
  3. Store the process start time in a global variable.
  4. Add a new periodically executed function to main.py (similar to the stats updates) that runs every 4h and updates the streak value but for the first time not before the process ran for more than 24h (e.g. to gracefully recover from potential system outages, simply return if timedelta to procses-start time < 24h). Set the streak_days value to 0 of all users who have a last_activity_date older than 24 hours and streak_days > 0. Increments the streak value by 1 and set streak_last_day_date to the current time for all user's having current_time() - streak_last_day_date > 24 h.

andreaskoepf avatar Jan 23 '23 20:01 andreaskoepf

@andreaskoepf i'll work on this

melvinebenezer avatar Jan 24 '23 15:01 melvinebenezer

Hello @andreaskoepf, @melvinebenezer,

We understand the need for a gamification layer to calculate the number of consecutive days on which users completed human feedback tasks and show them on the leaderboard. We agree with the proposed approach of removing streak_last_day_date & streak_days from UserStats and adding them to the User model, along with a nullable additional last_activity_date datetimetz column.

To improve upon this solution, we suggest adding a new feature that tracks the total number of tasks completed by a user in addition to the streak. This will give a more comprehensive view of a user's performance and allow for more meaningful comparisons on the leaderboard.

We also suggest setting up a separate cron job or a scheduled task to handle the update of the streak and total tasks completed. This would ensure that the updates are performed at regular intervals, even if the process encounters an outage.

In addition, we suggest the adding a new column that stores the time of the first task completion, which would be used to calculate the total number of days a user has been active on the platform. This would be a great way to show user loyalty and could be a great way to reward long-term users.

We are happy to work on implementing these suggestions and look forward to discussing this further.

Thanks, The Team

hemangjoshi37a avatar Jan 26 '23 19:01 hemangjoshi37a

@hemangjoshi37a Thanks for the suggestions. Parts of what you suggest are already implemented, e.g. the total score is calculated for the leaderboard-stats (incl. counts per job-type and different time-frames), periodic jobs for updating the user-stats are running.

For the user we currently only calculate the created_date (first login of the user). I am personally not sure if first-task-completion really adds much to this but if you want you can make a PR for it.

andreaskoepf avatar Jan 26 '23 19:01 andreaskoepf