cleaver icon indicating copy to clipboard operation
cleaver copied to clipboard

Track individual participation INCOMPLETE

Open ewheeler opened this issue 11 years ago • 0 comments

I'm making this PR just to have a place for discussion -- its incomplete because it does the RedisBackend and SQLAlchemyBackend behave differently at the moment. Also, depending on the vision/roadmap for cleaver, this may not be desired and can certainly live in a fork.

This PR adds the user identity as a parameter to cleaver API calls for mark_participant and mark_conversion Passing the identity allows the backend to keep track of individual participation so that experiments may be conducted in situations where conversion events might happen multiple times. Basically, the backend can keep track of the number of unique participants and score one and only one conversion for each unique participant. Otherwise, if the conversion event can happen more than once, its impossible to distinguish the true number of conversions for each variant (e.g., one individual scoring many conversions vs several individuals scoring several conversions) and therefore difficult to draw conclusions from the experiment results.

This different behavior could be enabled/disabled through an option passed to the backend so that cleaver defaults to the original behavior. It was pretty trivial to implement in the RedisBackend, but would require new models and some additional complications in the SQLAlchemyBackend

ewheeler avatar Aug 09 '13 21:08 ewheeler