core icon indicating copy to clipboard operation
core copied to clipboard

feat: Make a/b tests independent

Open MarioAslau opened this issue 2 months ago • 0 comments

Explanation

Right now, only the ID is used as a seed to determine what bucket the user falls into in an A/B test.

Due to this, if there’s more than 1 A/B test configured, they will never be independent of each other

Solution: Define the distribution using profile/metametrics ID + A/B test ID as a seed instead of just profile/metametrics ID

  • added createDeterministicSeed: Convert concatenated strings into valid hex format

References

Checklist

  • [ ] I've updated the test suite for new or updated code as appropriate
  • [ ] I've updated documentation (JSDoc, Markdown, etc.) for new or updated code as appropriate
  • [ ] I've communicated my changes to consumers by updating changelogs for packages I've changed
  • [ ] I've introduced breaking changes in this PR and have prepared draft pull requests for clients and consumer packages to resolve them

[!NOTE] Use a per-flag deterministic seed (metaMetricsId + flagName) for A/B selection, instead of a shared seed, to decouple experiments.

  • Remote feature flag processing (packages/remote-feature-flag-controller/src/remote-feature-flag-controller.ts)
    • A/B group selection now seeds generateDeterministicRandomNumber with metaMetricsId + remoteFeatureFlagName (per-flag) instead of a single shared seed.
    • Computes thresholdValue inside the per-flag array handling block and removes the truthy check, always evaluating selection for array-based flags.

Written by Cursor Bugbot for commit 4fa286e31fe942cdae604909e6480db66083a9fc. This will update automatically on new commits. Configure here.

MarioAslau avatar Dec 16 '25 03:12 MarioAslau