python-mapswipe-workers icon indicating copy to clipboard operation
python-mapswipe-workers copied to clipboard

Set reasonable factor for maximum time spent allowed on Street tasks for Community Dashboard stats

Open ofr1tz opened this issue 8 months ago • 2 comments

To my understanding, a maximum allowed time spent is applied to each mapping session when calculating the contribution times for the Community Dashboard. This maximum is calculated as number of tasks per mapping session multiplied by a factor that is project type specific (see here).

Currently, this factor is not defined for projects of type Street and thus defaults to 1. Therefore, we count a maximum of 1 second of mapping time per task for contributions to Street projects. This seems way to little for projects of this type that often demand for more thorough inspection of the imagery.

ofr1tz avatar Mar 18 '25 09:03 ofr1tz

For the other project types, the factor has been determined by calculating the 95th percentile of mapping time per task for (then) existing mapping sessions.

Currently, this would provide the following results:

 project_type |     median      |  95_percentile  |       avg       
--------------+-----------------+-----------------+-----------------
            1 | 00:00:00.246056 | 00:00:01.481481 | 00:00:24.659448
            2 | 00:00:01.731568 | 00:00:08.414459 | 00:00:04.738812
            3 | 00:00:02.045298 | 00:00:10.892354 | 00:00:05.691562
            7 | 00:00:10.711909 | 00:01:03.926002 | 00:00:27.789516

More than 1 minute maximum counted mapping time per Street task seems a bit much. The current result may be due to the fact that the project type is very new and users are not yet accustomed to it. It might be a good idea, however, to start with this very generous maximum, and decrease the value as soon as we have a better base for analysis.

ofr1tz avatar Mar 18 '25 12:03 ofr1tz

What happens when a shorter max time is set? Does it change users' all-time stats, or does it only impact things going forward from that point? Sometimes we need to rerun stats for a previous period (if the server was down and no one caught it, or if there's a fix to an error in how some stat was calculated)?

danbjoseph avatar Mar 18 '25 20:03 danbjoseph