ab_test returning unwanted result when using DualAdapter + allow_multiple_experiments

[Drowze]

Describe the bug When I have a DualAdapter set (tested with Cookies adapter for not logged in and Redis adapter for logged in) with fallback_to_logged_out_adapter set to true and allow_multiple_experiments set to control (possible to true as well), if a guest user is bucketed to a non-control version, he'll always view the control version once he logs in.

To Reproduce Steps to reproduce the behavior:

1. Given a configuration as follows:

Split.configure do |config|
  config.redis = Redis.current
  config.persistence = Split::Persistence::DualAdapter.with_config(
    fallback_to_logged_out_adapter: true,
    logged_in: ->(context) { context.current_client.logged? },
    logged_in_adapter: redis_adapter,
    logged_out_adapter: cookie_adapter
  )
  config.persistence_cookie_length = 30.days.to_i
  config.allow_multiple_experiments = 'control'
end

2. And given an user who has been bucketed to a non-control experiment version
3. When the user logs in, he'll always instead see the control version (and will not count on the admin dashboard)

Expected behaviour Given a DualAdapter configuration with fallback_to_logged_out_adapter set to true, I would expect that once the client logs in, he'd keep seeing the version he's seeing as a guest.

Additional context I just tested with allow_multiple_experiments set to control, but I suspect the result would be the same it's set to true. When it's set to false, however, it works fine according the expected behaviour.

[andrehjr]

Hi there @Drowze, I made a few tests both using allow_multiple_experiments as control/true. And it worked as expected to me. So, I was not able to reproduce this issue 🤔 any more details you can provide?

There was a bug fixed by https://github.com/splitrb/split/pull/613 that may be related to this, but it's only fixed on master.

Were experiments running for a long time?

[jmccaffrey]

I ran into a similar issue I believe. The goal was to track if showing a user that wasn't logged in a new feature, would lead them to logging in and completing a task. If you are using the dual adapter with cookie and redis, I don't think the system by default is going to understand that the not-logged-in user tracked by cookie A is the same logged-in-user tracked by current_user_id: 4533. I was only able to fully track across the login flow when sticking with only cookie or session. Would there be a way to track the user across that gap?

[TheRusskiy]

Also stumbled on this bug, when a user logs in PARTICIPANTS increments by 1, disabling allow_multiple_experiments fixes it, although it's not a proper fix for multiple experiments.

Briefly looking at gem's code couldn't the source of this problem.

Happens on the latest release version.