posthog icon indicating copy to clipboard operation
posthog copied to clipboard

fix(trend experiment): fix credible interval delta

Open jurajmajerik opened this issue 1 year ago â€ĸ 1 comments

Problem

Forgot to divide by the control mean to get the correct credible interval delta.

I didn't catch it because, by coincidence, the result still made sense in my local example. A user pointed it out here: https://posthog.slack.com/archives/C06KJQ2RH1A/p1724761173381449

How did you test this code?

👀

jurajmajerik avatar Aug 28 '24 10:08 jurajmajerik

Size Change: 0 B

Total Size: 1.08 MB

â„šī¸ View Unchanged
Filename Size
frontend/dist/toolbar.js 1.08 MB

compressed-size-action

github-actions[bot] avatar Aug 28 '24 10:08 github-actions[bot]

Added a comment + smuggling two more changes, as users reported these as confusing:

  • the summary now mentions the win percentage for both Trends and Funnels, making it consistent
  • the credible interval for Funnel is now also expressed w.r.t. %Delta, same like Trends

jurajmajerik avatar Aug 28 '24 14:08 jurajmajerik

📸 UI snapshots have been updated

2 snapshot changes in total. 0 added, 2 modified, 0 deleted:

  • chromium: 0 added, 2 modified, 0 deleted (diff for shard 2)
  • webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

posthog-bot avatar Aug 28 '24 14:08 posthog-bot