oTree icon indicating copy to clipboard operation
oTree copied to clipboard

Automatically doubling the number of players for mTurk session is problematic

Open hashemamireh opened this issue 5 years ago • 2 comments

The automatic functionality where the number of players is automatically doubled when an mTurk session is created, causes problems for certain experiments. Due to grouping schemes, certain experiments need a specific number of players. And the number of players must exactly match the number of HITs. Forcing automatic doubling of the number of players makes no sense in these cases. It makes more sense to have that as an option rather than an unchangeable default setting.

hashemamireh avatar Sep 26 '20 20:09 hashemamireh

Thank you for the feedback.

And the number of players must exactly match the number of HITs.

but what if a worker fails to complete the HIT in time? Then the HIT will be returned, and if there are no spares, then all the subsequent workers who try the HIT will get an error page since there are no spare participant links.

oTree-org avatar Sep 26 '20 21:09 oTree-org

but what if a worker fails to complete the HIT in time? Then the HIT will be returned, and if there are no spares, then all the subsequent workers who try the HIT will get an error page since there are no spare participant links.

I honestly didn't think of this until you mentioned it here. Would it be possible that if someone "returns" their HIT (or lets it expire), that it does not get listed again? I checked the mTurk documentation briefly and it doesn't seems to say anything about this explicitly. I'm definitely gonna be giving this more thought and if I can come up with anything, I'll definitely post here again. That said, one (not ideal) solution would be that when someone clicks the link on a "recycled HIT" and there aren't any spare links, they get an error message saying that the HIT is no longer valid and that they should "return" it.

If there really isn't a way around this, it might be a good idea to update the oTree documentation (Groups section and mTurk section) to explain how this functionality might be incompatible with certain grouping designs.

hashemamireh avatar Sep 26 '20 21:09 hashemamireh