python-mapswipe-workers
python-mapswipe-workers copied to clipboard
Set reasonable factor for maximum time spent allowed on Street tasks for Community Dashboard stats
To my understanding, a maximum allowed time spent is applied to each mapping session when calculating the contribution times for the Community Dashboard. This maximum is calculated as number of tasks per mapping session multiplied by a factor that is project type specific (see here).
Currently, this factor is not defined for projects of type Street and thus defaults to 1. Therefore, we count a maximum of 1 second of mapping time per task for contributions to Street projects. This seems way to little for projects of this type that often demand for more thorough inspection of the imagery.
For the other project types, the factor has been determined by calculating the 95th percentile of mapping time per task for (then) existing mapping sessions.
Currently, this would provide the following results:
project_type | median | 95_percentile | avg
--------------+-----------------+-----------------+-----------------
1 | 00:00:00.246056 | 00:00:01.481481 | 00:00:24.659448
2 | 00:00:01.731568 | 00:00:08.414459 | 00:00:04.738812
3 | 00:00:02.045298 | 00:00:10.892354 | 00:00:05.691562
7 | 00:00:10.711909 | 00:01:03.926002 | 00:00:27.789516
More than 1 minute maximum counted mapping time per Street task seems a bit much. The current result may be due to the fact that the project type is very new and users are not yet accustomed to it. It might be a good idea, however, to start with this very generous maximum, and decrease the value as soon as we have a better base for analysis.
What happens when a shorter max time is set? Does it change users' all-time stats, or does it only impact things going forward from that point? Sometimes we need to rerun stats for a previous period (if the server was down and no one caught it, or if there's a fix to an error in how some stat was calculated)?