[DO NOT MERGE] AB Testing POC

[cemms1]

AB Testing PoC Branch

What

The Commercial Development team are looking into revamping our AB testing setup

Our RFC document is here for more information

This proposal makes use of Fastly dictionaries to store both our AB testing metadata and audience segmentation for AB tests

Why

The current AB testing setup has a number of problems so we'd like to attempt to improve it

Pull requests

Includes work from the following PRs:

• https://github.com/guardian/dotcom-rendering/pull/13861
• https://github.com/guardian/dotcom-rendering/pull/13823
• https://github.com/guardian/dotcom-rendering/pull/13849
• https://github.com/guardian/dotcom-rendering/pull/13897
• https://github.com/guardian/dotcom-rendering/pull/13994

Next steps

• Review/Discussion of the PoC with other teams
• Add deploy keys for CODE fastly service, allowing the ab test dictionaries to start to be updated in CODE fastly service (which can be deployed with AB testing VCL) from this PR or main.

[github-actions[bot]]

Deploy build 44272 of Mobile::mobile-apps-rendering to CODE

All deployment options

• Deploy build 44272 of Mobile::mobile-apps-rendering to CODE
• Deploy parts of build 44272 to CODE by previewing it first
• What's on CODE right now?

---

From guardian/actions-riff-raff.

[github-actions[bot]]

Deploy build 20047 of dotcom:rendering-all to CODE

All deployment options

• Deploy build 20047 of dotcom:rendering-all to CODE
• Deploy parts of build 20047 to CODE by previewing it first
• What's on CODE right now?

---

From guardian/actions-riff-raff.

[github-actions[bot]]

Size Change: 0 B

Total Size: 1.04 MB

ℹ️ View Unchanged

Filename	Size
dotcom-rendering/dist/1160.client.web.********************.js	3.56 kB
dotcom-rendering/dist/1212.client.web.********************.js	7.95 kB
dotcom-rendering/dist/1521.client.web.********************.js	2.89 kB
dotcom-rendering/dist/1695.client.web.********************.js	3.39 kB
dotcom-rendering/dist/1931.client.web.********************.js	3.43 kB
dotcom-rendering/dist/2244.client.web.********************.js	3.56 kB
dotcom-rendering/dist/2373.client.web.********************.js	4.52 kB
dotcom-rendering/dist/2400.client.web.********************.js	2.11 kB
dotcom-rendering/dist/2756.client.web.********************.js	5.4 kB
dotcom-rendering/dist/2909.client.web.********************.js	20.4 kB
dotcom-rendering/dist/3365.client.web.********************.js	3.27 kB
dotcom-rendering/dist/3944.client.web.********************.js	2.51 kB
dotcom-rendering/dist/3979.client.web.********************.js	4.18 kB
dotcom-rendering/dist/420.client.web.********************.js	4.35 kB
dotcom-rendering/dist/4312.client.web.********************.js	3.07 kB
dotcom-rendering/dist/4501.client.web.********************.js	4.29 kB
dotcom-rendering/dist/4524.client.web.********************.js	3.62 kB
dotcom-rendering/dist/4753.client.web.********************.js	5.64 kB
dotcom-rendering/dist/4754.client.web.********************.js	7.66 kB
dotcom-rendering/dist/4840.client.web.********************.js	3.67 kB
dotcom-rendering/dist/5351.client.web.********************.js	51.5 kB
dotcom-rendering/dist/5423.client.web.********************.js	2.76 kB
dotcom-rendering/dist/5549.client.web.********************.js	3.9 kB
dotcom-rendering/dist/5709.client.web.********************.js	16.5 kB
dotcom-rendering/dist/5895.client.web.********************.js	22.7 kB
dotcom-rendering/dist/5901.client.web.********************.js	4.21 kB
dotcom-rendering/dist/5980.client.web.********************.js	5.43 kB
dotcom-rendering/dist/6085.client.web.********************.js	530 B
dotcom-rendering/dist/6232.client.web.********************.js	618 B
dotcom-rendering/dist/6255.client.web.********************.js	12.1 kB
dotcom-rendering/dist/6283.client.web.********************.js	3.9 kB
dotcom-rendering/dist/6316.client.web.********************.js	5.04 kB
dotcom-rendering/dist/6317.client.web.********************.js	4.23 kB
dotcom-rendering/dist/6369.client.web.********************.js	4.78 kB
dotcom-rendering/dist/6504.client.web.********************.js	3.29 kB
dotcom-rendering/dist/6545.client.web.********************.js	6.01 kB
dotcom-rendering/dist/6683.client.web.********************.js	2.87 kB
dotcom-rendering/dist/6694.client.web.********************.js	2.76 kB
dotcom-rendering/dist/6721.client.web.********************.js	3.2 kB
dotcom-rendering/dist/6832.client.web.********************.js	68 kB
dotcom-rendering/dist/7008.client.web.********************.js	4.7 kB
dotcom-rendering/dist/7028.client.web.********************.js	3.17 kB
dotcom-rendering/dist/7116.client.web.********************.js	23.2 kB
dotcom-rendering/dist/736.client.web.********************.js	3.8 kB
dotcom-rendering/dist/739.client.web.********************.js	528 B
dotcom-rendering/dist/7416.client.web.********************.js	2.87 kB
dotcom-rendering/dist/7501.client.web.********************.js	3.63 kB
dotcom-rendering/dist/7819.client.web.********************.js	3.86 kB
dotcom-rendering/dist/7825.client.web.********************.js	3.77 kB
dotcom-rendering/dist/7903.client.web.********************.js	10.4 kB
dotcom-rendering/dist/7986.client.web.********************.js	3.03 kB
dotcom-rendering/dist/8110.client.web.********************.js	3.48 kB
dotcom-rendering/dist/8158.client.web.********************.js	157 B
dotcom-rendering/dist/8361.client.web.********************.js	3.44 kB
dotcom-rendering/dist/8389.client.web.********************.js	2.63 kB
dotcom-rendering/dist/8393.client.web.********************.js	3.34 kB
dotcom-rendering/dist/8406.client.web.********************.js	438 B
dotcom-rendering/dist/8569.client.web.********************.js	5.04 kB
dotcom-rendering/dist/8742.client.web.********************.js	4.86 kB
dotcom-rendering/dist/8903.client.web.********************.js	20.5 kB
dotcom-rendering/dist/8938.client.web.********************.js	4.49 kB
dotcom-rendering/dist/8957.client.web.********************.js	19.9 kB
dotcom-rendering/dist/9193.client.web.********************.js	5.1 kB
dotcom-rendering/dist/9230.client.web.********************.js	10.5 kB
dotcom-rendering/dist/927.client.web.********************.js	2.75 kB
dotcom-rendering/dist/939.client.web.********************.js	4.53 kB
dotcom-rendering/dist/Accessibility-importable.client.web.********************.js	6.64 kB
dotcom-rendering/dist/AdBlockAsk-importable.client.web.********************.js	2.99 kB
dotcom-rendering/dist/AdPortals-importable.client.web.********************.js	4.81 kB
dotcom-rendering/dist/AlreadyVisited-importable.client.web.********************.js	425 B
dotcom-rendering/dist/AppsEpic-importable.client.web.********************.js	3.67 kB
dotcom-rendering/dist/AppsFooter-importable.client.web.********************.js	2.75 kB
dotcom-rendering/dist/AppsLightboxImage-importable.client.web.********************.js	3.06 kB
dotcom-rendering/dist/AppsLightboxImageStore-importable.client.web.********************.js	2.65 kB
dotcom-rendering/dist/AudioAtomWrapper-importable.client.web.********************.js	2.76 kB
dotcom-rendering/dist/AudioPlayerWrapper-importable.client.web.********************.js	6.61 kB
dotcom-rendering/dist/AustralianTerritorySwitcher-importable.client.web.********************.js	1.99 kB
dotcom-rendering/dist/Branding-importable.client.web.********************.js	2.87 kB
dotcom-rendering/dist/braze-web-sdk-core.client.web.********************.js	67.7 kB
dotcom-rendering/dist/BrazeMessaging-importable.client.web.********************.js	1.69 kB
dotcom-rendering/dist/CalloutBlockComponent-importable.client.web.********************.js	6.74 kB
dotcom-rendering/dist/CalloutEmbedBlockComponent-importable.client.web.********************.js	5.77 kB
dotcom-rendering/dist/CardCommentCount-importable.client.web.********************.js	2.66 kB
dotcom-rendering/dist/Carousel-importable.client.web.********************.js	6.67 kB
dotcom-rendering/dist/CarouselForNewsletters-importable.client.web.********************.js	4.57 kB
dotcom-rendering/dist/ChartAtom-importable.client.web.********************.js	540 B
dotcom-rendering/dist/CommentCount-importable.client.web.********************.js	2.28 kB
dotcom-rendering/dist/CrosswordComponent-importable.client.web.********************.js	2.87 kB
dotcom-rendering/dist/DiscussionApps-importable.client.web.********************.js	2 kB
dotcom-rendering/dist/DiscussionMeta-importable.client.web.********************.js	2.39 kB
dotcom-rendering/dist/DiscussionWeb-importable.client.web.********************.js	1.69 kB
dotcom-rendering/dist/DocumentBlockComponent-importable.client.web.********************.js	2.85 kB
dotcom-rendering/dist/Dropdown-importable.client.web.********************.js	1.72 kB
dotcom-rendering/dist/EditionSwitcherBanner-importable.client.web.********************.js	4.43 kB
dotcom-rendering/dist/EmbedBlockComponent-importable.client.web.********************.js	3.97 kB
dotcom-rendering/dist/EnhanceAffiliateLinks-importable.client.web.********************.js	503 B
dotcom-rendering/dist/EnhancePinnedPost-importable.client.web.********************.js	2.02 kB
dotcom-rendering/dist/FetchOnwardsData-importable.client.web.********************.js	1.72 kB
dotcom-rendering/dist/FilterKeyEventsToggle-importable.client.web.********************.js	3.71 kB
dotcom-rendering/dist/FocusStyles-importable.client.web.********************.js	619 B
dotcom-rendering/dist/FollowWrapper-importable.client.web.********************.js	2.51 kB
dotcom-rendering/dist/FootballMatchesPageWrapper-importable.client.web.********************.js	8.17 kB
dotcom-rendering/dist/FootballTablesCompetitionSelect-importable.client.web.********************.js	3.27 kB
dotcom-rendering/dist/FooterLabel-importable.client.web.********************.js	364 B
dotcom-rendering/dist/FooterReaderRevenueLinks-importable.client.web.********************.js	3.41 kB
dotcom-rendering/dist/frameworks.client.web.********************.js	20.9 kB
dotcom-rendering/dist/FrontSubNav-importable.client.web.********************.js	7.46 kB
dotcom-rendering/dist/GetCricketScoreboard-importable.client.web.********************.js	6.24 kB
dotcom-rendering/dist/GetMatchNav-importable.client.web.********************.js	12 kB
dotcom-rendering/dist/GetMatchStats-importable.client.web.********************.js	8.06 kB
dotcom-rendering/dist/GetMatchTabs-importable.client.web.********************.js	2.57 kB
dotcom-rendering/dist/guardian-braze-components-banner.client.web.********************.js	16.3 kB
dotcom-rendering/dist/guardian-braze-components-end-of-article.client.web.********************.js	10.2 kB
dotcom-rendering/dist/GuideAtomWrapper-importable.client.web.********************.js	779 B
dotcom-rendering/dist/index.client.web.********************.js	46.6 kB
dotcom-rendering/dist/InstagramBlockComponent-importable.client.web.********************.js	2.89 kB
dotcom-rendering/dist/InteractiveAtomMessenger-importable.client.web.********************.js	851 B
dotcom-rendering/dist/InteractiveBlockComponent-importable.client.web.********************.js	8.77 kB
dotcom-rendering/dist/InteractiveContentsBlockComponent-importable.client.web.********************.js	3.79 kB
dotcom-rendering/dist/InteractivesDisableArticleSwipe-importable.client.web.********************.js	2.21 kB
dotcom-rendering/dist/InteractivesNativePlatformWrapper-importable.client.web.********************.js	3.2 kB
dotcom-rendering/dist/KeyEventsCarousel-importable.client.web.********************.js	5.75 kB
dotcom-rendering/dist/KnowledgeQuizAtom-importable.client.web.********************.js	3.23 kB
dotcom-rendering/dist/LatestLinks-importable.client.web.********************.js	7.82 kB
dotcom-rendering/dist/LightboxHash-importable.client.web.********************.js	436 B
dotcom-rendering/dist/LightboxLayout-importable.client.web.********************.js	6.58 kB
dotcom-rendering/dist/ListenToArticle-importable.client.web.********************.js	4.34 kB
dotcom-rendering/dist/LiveBlogEpic-importable.client.web.********************.js	3.58 kB
dotcom-rendering/dist/LiveblogGutterAskWrapper-importable.client.web.********************.js	2.51 kB
dotcom-rendering/dist/LiveblogNotifications-importable.client.web.********************.js	4.89 kB
dotcom-rendering/dist/Liveness-importable.client.web.********************.js	4.7 kB
dotcom-rendering/dist/LoopVideo-importable.client.web.********************.js	6.1 kB
dotcom-rendering/dist/ManyNewsletterSignUp-importable.client.web.********************.js	7.65 kB
dotcom-rendering/dist/MapEmbedBlockComponent-importable.client.web.********************.js	6.03 kB
dotcom-rendering/dist/Metrics-importable.client.web.********************.js	2.71 kB
dotcom-rendering/dist/MostViewedFooter-importable.client.web.********************.js	3.85 kB
dotcom-rendering/dist/MostViewedFooterData-importable.client.web.********************.js	5.95 kB
dotcom-rendering/dist/MostViewedRightWithAd-importable.client.web.********************.js	5.24 kB
dotcom-rendering/dist/OnwardsUpper-importable.client.web.********************.js	5.11 kB
dotcom-rendering/dist/PersonalityQuizAtom-importable.client.web.********************.js	3.36 kB
dotcom-rendering/dist/ProfileAtom-importable.client.web.********************.js	540 B
dotcom-rendering/dist/ProfileAtomWrapper-importable.client.web.********************.js	801 B
dotcom-rendering/dist/PulsingDot-importable.client.web.********************.js	750 B
dotcom-rendering/dist/QandaAtom-importable.client.web.********************.js	538 B
dotcom-rendering/dist/ReaderRevenueDev-importable.client.web.********************.js	470 B
dotcom-rendering/dist/readerRevenueDevUtils.client.web.********************.js	1.7 kB
dotcom-rendering/dist/RelativeTime-importable.client.web.********************.js	2.54 kB
dotcom-rendering/dist/RichLinkComponent-importable.client.web.********************.js	6.15 kB
dotcom-rendering/dist/ScrollableFeature-importable.client.web.********************.js	6.76 kB
dotcom-rendering/dist/ScrollableHighlights-importable.client.web.********************.js	7.2 kB
dotcom-rendering/dist/ScrollableMedium-importable.client.web.********************.js	2.17 kB
dotcom-rendering/dist/ScrollableSmall-importable.client.web.********************.js	2.24 kB
dotcom-rendering/dist/SecureSignup-importable.client.web.********************.js	4.21 kB
dotcom-rendering/dist/SendTargetingParams-importable.client.web.********************.js	2.27 kB
dotcom-rendering/dist/sentry.client.web.********************.js	803 B
dotcom-rendering/dist/SetABTests-importable.client.web.********************.js	3.93 kB
dotcom-rendering/dist/SetAdTargeting-importable.client.web.********************.js	487 B
dotcom-rendering/dist/ShareButton-importable.client.web.********************.js	2.17 kB
dotcom-rendering/dist/shimport.client.web.********************.js	2.8 kB
dotcom-rendering/dist/ShowHideContainers-importable.client.web.********************.js	874 B
dotcom-rendering/dist/ShowMore-importable.client.web.********************.js	1.68 kB
dotcom-rendering/dist/SignInGateMain.client.web.********************.js	1.11 kB
dotcom-rendering/dist/SignInGateMainCheckoutComplete.client.web.********************.js	2.6 kB
dotcom-rendering/dist/SignInGateSelector-importable.client.web.********************.js	5.06 kB
dotcom-rendering/dist/SlideshowCarousel-importable.client.web.********************.js	4.51 kB
dotcom-rendering/dist/SlotBodyEnd-importable.client.web.********************.js	4.9 kB
dotcom-rendering/dist/SpotifyBlockComponent-importable.client.web.********************.js	5.77 kB
dotcom-rendering/dist/StickyBottomBanner-importable.client.web.********************.js	6.08 kB
dotcom-rendering/dist/SubNav-importable.client.web.********************.js	2.42 kB
dotcom-rendering/dist/TableOfContents-importable.client.web.********************.js	3.65 kB
dotcom-rendering/dist/TimelineAtom-importable.client.web.********************.js	1.23 kB
dotcom-rendering/dist/Titlepiece-importable.client.web.********************.js	13.6 kB
dotcom-rendering/dist/TopBar-importable.client.web.********************.js	8.82 kB
dotcom-rendering/dist/TopBarSupport-importable.client.web.********************.js	2.5 kB
dotcom-rendering/dist/TweetBlockComponent-importable.client.web.********************.js	1.13 kB
dotcom-rendering/dist/UnsafeEmbedBlockComponent-importable.client.web.********************.js	2.9 kB
dotcom-rendering/dist/VideoFacebookBlockComponent-importable.client.web.********************.js	6.04 kB
dotcom-rendering/dist/VineBlockComponent-importable.client.web.********************.js	3.33 kB
dotcom-rendering/dist/YoutubeBlockComponent-importable.client.web.********************.js	843 B

_{compressed-size-action}

[Jakeii]

Notes from 17/04/2025:

• Need to outline how test expiration will work
• use test groups instead of controlGroup/testGroups
• can we have a way for analysts to see the tests and state, via frontend or github pages
• 0% tests how will they work?
• Opting in/out of tests

[Jakeii]

Notes from 24/04/2025:

• We don't think we want/need explicit variant sizing, would like to size the whole test based on the overall test size.
• no downside to expanding to 1000 mvtids, and will allow to split for example a 10% test into 3 x 3.3% variants if ever needed
• have the date be a string, less boilerplate

[github-actions[bot]]

"This PR is stale because it has been open 30 days with no activity. Unless a comment is added or the “stale” label removed, this will be closed in 3 days"

[Jakeii]

This Proof of Concept PR has be superseded by the actual implementation PRs #14333, #14344 and #14404