feat: Don't disable Turbosnap when package.json changes

[AshCorr]

(Sorry about the long blurb of HIGHLY confusing Chromatic details)

What does this change?

Changes our Chromatic action to keep Turbosnap enabled even if it detects a change to the package.json, except on Dependency PR's.

August Figures

Total Snapshots Taken: 910,000 Total Snapshots Skipped: 950,000 Overall Snapshots Taken/Skipped: 1,860,000

• 100,000 Snapshots generated by dependabot PR's where Turbosnap is disabled.

  • Not much we can do about those, it's useful to have visual regression testing on dependency changes. Chromatic have mentioned that they're working on making Turbosnap a bit more intelligent so that it can understand the impacts of a package.json change.

• 150,000 Snapshots generated on the Main branch with Turbosnap disabled.

  • Not useful at all, changes on main get auto accepted, even if Chromatic did find an issue caused by a dependency change it wouldn't be flagged for review.

  • Most of the time this happens when Chromatic runs on a merge commit in main. It's hard to describe exactly whats going on here but hopefully this chart helps:

    Feature Branch created from main->Dependabot PR merged into main->Feature Branch merged into main->Turbosnap disabled on merge commit between main and feature branch due to the dependency changes in main

    From the Turbosnap docs:

    When you have a merge commit, Chromatic considers any file that has changed since either ancestor’s commit to decide if a story needs to be re-snapshotted. In other words, the union of the git changes.

• 350,000 Snapshots generated by non main Branches with Turbosnap disabled

  • It was really hard to pinpoint exactly whats causing Turbosnap to be disabled for these builds, but I think I've narrowed it mostly down to 2 things:

    1. Merging main into a branch, main might have dependency changes which disable Turbosnap.

    2. Squashed commits. I found this one really tricky to investigate and I'm still not sure if I've wrapped my head around it or not. Here's the general gist of what I understood.

       When PR gets squashed into a single commit(Before merging - I'm not talking about a squash merge) Turbosnap loses some critical data about the commit history used for working out changed files. It resolves this by finding the commit which the branch was created from and then copies build information from the last Chromatic build that ran on that commit. Since we don't commit directly to main our branches are always branching off a merge commit (Most recent commit on main will always be a merge commit), this means that the build that Chromatic copies data from is always a Merge commit build and seems to come with a lot of the other issues mentioned with Merge commits earlier.

Other things worth trying

As of chromatic 6.6.* Turbosnap supports using the Rebase option when merging a PR. Since this type of merge doesn't create a merge commit I wonder if we could improve Turbosnap perfomance by enforcing this merge strategy on PR's.

Why?

Less snapshots = Less false positives & smaller bills💰💰💰

[github-actions[bot]]

Size Change: +170 B (0%)

Total Size: 1.68 MB

Filename	Size	Change
dotcom-rendering/dist/frontend.server.js	506 kB	+170 B (0%)

ℹ️ View Unchanged

Filename	Size
dotcom-rendering/dist/1382.legacy.********************.js	5.69 kB
dotcom-rendering/dist/1794.legacy.********************.js	3.32 kB
dotcom-rendering/dist/2344.********************.js	2.72 kB
dotcom-rendering/dist/2344.legacy.********************.js	2.78 kB
dotcom-rendering/dist/2486.legacy.********************.js	4.54 kB
dotcom-rendering/dist/259.legacy.********************.js	3.25 kB
dotcom-rendering/dist/2947.********************.js	2.74 kB
dotcom-rendering/dist/2947.legacy.********************.js	2.82 kB
dotcom-rendering/dist/2959.legacy.********************.js	4.92 kB
dotcom-rendering/dist/3210.********************.js	8.6 kB
dotcom-rendering/dist/3307.********************.js	3.12 kB
dotcom-rendering/dist/3584.********************.js	1.8 kB
dotcom-rendering/dist/3584.legacy.********************.js	1.8 kB
dotcom-rendering/dist/3592.********************.js	20.9 kB
dotcom-rendering/dist/3592.legacy.********************.js	21.8 kB
dotcom-rendering/dist/3872.legacy.********************.js	30.1 kB
dotcom-rendering/dist/4019.********************.js	3.44 kB
dotcom-rendering/dist/4019.legacy.********************.js	3.48 kB
dotcom-rendering/dist/4071.legacy.********************.js	2.8 kB
dotcom-rendering/dist/4541.legacy.********************.js	5.39 kB
dotcom-rendering/dist/4869.legacy.********************.js	6.7 kB
dotcom-rendering/dist/5058.********************.js	2.73 kB
dotcom-rendering/dist/5553.********************.js	2.87 kB
dotcom-rendering/dist/5887.********************.js	5.94 kB
dotcom-rendering/dist/6131.********************.js	4.29 kB
dotcom-rendering/dist/6131.legacy.********************.js	4.3 kB
dotcom-rendering/dist/6165.********************.js	33.1 kB
dotcom-rendering/dist/6204.********************.js	5.56 kB
dotcom-rendering/dist/6400.********************.js	21.5 kB
dotcom-rendering/dist/6400.legacy.********************.js	21.5 kB
dotcom-rendering/dist/6462.legacy.********************.js	4.16 kB
dotcom-rendering/dist/6887.legacy.********************.js	3.54 kB
dotcom-rendering/dist/7148.********************.js	6.8 kB
dotcom-rendering/dist/7148.legacy.********************.js	9.68 kB
dotcom-rendering/dist/7348.legacy.********************.js	6.12 kB
dotcom-rendering/dist/7576.********************.js	3.96 kB
dotcom-rendering/dist/7576.legacy.********************.js	5.38 kB
dotcom-rendering/dist/7800.********************.js	11.3 kB
dotcom-rendering/dist/7829.********************.js	2.8 kB
dotcom-rendering/dist/7829.legacy.********************.js	2.91 kB
dotcom-rendering/dist/7981.********************.js	4.32 kB
dotcom-rendering/dist/8109.********************.js	6.79 kB
dotcom-rendering/dist/8129.legacy.********************.js	11.8 kB
dotcom-rendering/dist/8195.legacy.********************.js	2.98 kB
dotcom-rendering/dist/8414.********************.js	4.38 kB
dotcom-rendering/dist/8414.legacy.********************.js	4.46 kB
dotcom-rendering/dist/8471.legacy.********************.js	5.45 kB
dotcom-rendering/dist/8492.legacy.********************.js	4.41 kB
dotcom-rendering/dist/8612.********************.js	3.69 kB
dotcom-rendering/dist/8612.legacy.********************.js	4.18 kB
dotcom-rendering/dist/8789.legacy.********************.js	2.8 kB
dotcom-rendering/dist/8808.********************.js	5.3 kB
dotcom-rendering/dist/9154.********************.js	3.19 kB
dotcom-rendering/dist/9154.legacy.********************.js	3.3 kB
dotcom-rendering/dist/9380.legacy.********************.js	2.83 kB
dotcom-rendering/dist/9748.********************.js	4.74 kB
dotcom-rendering/dist/AlreadyVisited-importable.********************.js	4.52 kB
dotcom-rendering/dist/AlreadyVisited-importable.legacy.********************.js	4.52 kB
dotcom-rendering/dist/atomIframe.********************.js	757 B
dotcom-rendering/dist/atomIframe.legacy.********************.js	806 B
dotcom-rendering/dist/AudioAtomWrapper-importable.********************.js	461 B
dotcom-rendering/dist/AudioAtomWrapper-importable.legacy.********************.js	518 B
dotcom-rendering/dist/bootCmp.********************.js	9.52 kB
dotcom-rendering/dist/bootCmp.legacy.********************.js	13.3 kB
dotcom-rendering/dist/Branding-importable.********************.js	4.65 kB
dotcom-rendering/dist/Branding-importable.legacy.********************.js	4.65 kB
dotcom-rendering/dist/braze-web-sdk-core.********************.js	36.9 kB
dotcom-rendering/dist/braze-web-sdk-core.legacy.********************.js	36.9 kB
dotcom-rendering/dist/BrazeMessaging-importable.********************.js	9.46 kB
dotcom-rendering/dist/BrazeMessaging-importable.legacy.********************.js	10.2 kB
dotcom-rendering/dist/CalloutBlockComponent-importable.********************.js	4.26 kB
dotcom-rendering/dist/CalloutBlockComponent-importable.legacy.********************.js	4.55 kB
dotcom-rendering/dist/Carousel-importable.********************.js	11.5 kB
dotcom-rendering/dist/Carousel-importable.legacy.********************.js	12 kB
dotcom-rendering/dist/ChartAtomWrapper-importable.********************.js	261 B
dotcom-rendering/dist/ChartAtomWrapper-importable.legacy.********************.js	273 B
dotcom-rendering/dist/CommentCount-importable.********************.js	3.46 kB
dotcom-rendering/dist/CommentCount-importable.legacy.********************.js	3.56 kB
dotcom-rendering/dist/CommercialMetrics-importable.********************.js	8.97 kB
dotcom-rendering/dist/CommercialMetrics-importable.legacy.********************.js	813 B
dotcom-rendering/dist/CoreVitals-importable.********************.js	6.09 kB
dotcom-rendering/dist/CoreVitals-importable.legacy.********************.js	8.75 kB
dotcom-rendering/dist/debug.js	1.75 kB
dotcom-rendering/dist/DiscussionContainer-importable.********************.js	3.5 kB
dotcom-rendering/dist/DiscussionContainer-importable.legacy.********************.js	3.75 kB
dotcom-rendering/dist/DiscussionMeta-importable.********************.js	3.94 kB
dotcom-rendering/dist/DiscussionMeta-importable.legacy.********************.js	4.05 kB
dotcom-rendering/dist/DocumentBlockComponent-importable.********************.js	2.99 kB
dotcom-rendering/dist/DocumentBlockComponent-importable.legacy.********************.js	3.1 kB
dotcom-rendering/dist/dynamicImport.********************.js	2 kB
dotcom-rendering/dist/dynamicImport.legacy.********************.js	2.07 kB
dotcom-rendering/dist/EditionDropdown-importable.********************.js	4.1 kB
dotcom-rendering/dist/EditionDropdown-importable.legacy.********************.js	4.15 kB
dotcom-rendering/dist/EmbedBlockComponent-importable.********************.js	3.34 kB
dotcom-rendering/dist/EmbedBlockComponent-importable.legacy.********************.js	3.45 kB
dotcom-rendering/dist/embedIframe.********************.js	759 B
dotcom-rendering/dist/embedIframe.legacy.********************.js	810 B
dotcom-rendering/dist/EnhancePinnedPost-importable.********************.js	6.24 kB
dotcom-rendering/dist/EnhancePinnedPost-importable.legacy.********************.js	6.82 kB
dotcom-rendering/dist/FetchCommentCounts-importable.********************.js	1.73 kB
dotcom-rendering/dist/FetchCommentCounts-importable.legacy.********************.js	1.78 kB
dotcom-rendering/dist/FetchOnwardsData-importable.********************.js	2.23 kB
dotcom-rendering/dist/FetchOnwardsData-importable.legacy.********************.js	2.23 kB
dotcom-rendering/dist/FilterButton-importable.********************.js	2.64 kB
dotcom-rendering/dist/FilterButton-importable.legacy.********************.js	2.74 kB
dotcom-rendering/dist/FilterKeyEventsToggle-importable.********************.js	3.82 kB
dotcom-rendering/dist/FilterKeyEventsToggle-importable.legacy.********************.js	3.89 kB
dotcom-rendering/dist/FocusStyles-importable.********************.js	4.71 kB
dotcom-rendering/dist/FocusStyles-importable.legacy.********************.js	4.77 kB
dotcom-rendering/dist/ga.********************.js	2.83 kB
dotcom-rendering/dist/ga.legacy.********************.js	2.9 kB
dotcom-rendering/dist/GetCricketScoreboard-importable.********************.js	4.01 kB
dotcom-rendering/dist/GetCricketScoreboard-importable.legacy.********************.js	4.19 kB
dotcom-rendering/dist/GetMatchNav-importable.********************.js	7.36 kB
dotcom-rendering/dist/GetMatchNav-importable.legacy.********************.js	7.49 kB
dotcom-rendering/dist/GetMatchStats-importable.********************.js	6.36 kB
dotcom-rendering/dist/GetMatchStats-importable.legacy.********************.js	7.13 kB
dotcom-rendering/dist/GetMatchTabs-importable.********************.js	3.08 kB
dotcom-rendering/dist/GetMatchTabs-importable.legacy.********************.js	3.24 kB
dotcom-rendering/dist/guardian-braze-components-banner.********************.js	11.3 kB
dotcom-rendering/dist/guardian-braze-components-banner.js	10.3 kB
dotcom-rendering/dist/guardian-braze-components-banner.legacy.********************.js	11.4 kB
dotcom-rendering/dist/guardian-braze-components-end-of-article.********************.js	9.28 kB
dotcom-rendering/dist/guardian-braze-components-end-of-article.js	9.4 kB
dotcom-rendering/dist/guardian-braze-components-end-of-article.legacy.********************.js	9.36 kB
dotcom-rendering/dist/GuideAtomWrapper-importable.********************.js	263 B
dotcom-rendering/dist/GuideAtomWrapper-importable.legacy.********************.js	275 B
dotcom-rendering/dist/initDiscussion.********************.js	11.1 kB
dotcom-rendering/dist/initDiscussion.legacy.********************.js	11.5 kB
dotcom-rendering/dist/InstagramBlockComponent-importable.********************.js	3 kB
dotcom-rendering/dist/InstagramBlockComponent-importable.legacy.********************.js	3.09 kB
dotcom-rendering/dist/InteractiveBlockComponent-importable.********************.js	4.26 kB
dotcom-rendering/dist/InteractiveBlockComponent-importable.legacy.********************.js	4.43 kB
dotcom-rendering/dist/islands.********************.js	11.3 kB
dotcom-rendering/dist/islands.legacy.********************.js	12.2 kB
dotcom-rendering/dist/KeyEventsCarousel-importable.********************.js	2.9 kB
dotcom-rendering/dist/KeyEventsCarousel-importable.legacy.********************.js	3.01 kB
dotcom-rendering/dist/KnowledgeQuizAtomWrapper-importable.********************.js	269 B
dotcom-rendering/dist/KnowledgeQuizAtomWrapper-importable.legacy.********************.js	282 B
dotcom-rendering/dist/LabsHeader-importable.********************.js	6.56 kB
dotcom-rendering/dist/LabsHeader-importable.legacy.********************.js	6.69 kB
dotcom-rendering/dist/Links-importable.********************.js	6.33 kB
dotcom-rendering/dist/Links-importable.legacy.********************.js	7.39 kB
dotcom-rendering/dist/LiveBlogEpic-importable.********************.js	6.22 kB
dotcom-rendering/dist/LiveBlogEpic-importable.legacy.********************.js	6.9 kB
dotcom-rendering/dist/Liveness-importable.********************.js	3.58 kB
dotcom-rendering/dist/Liveness-importable.legacy.********************.js	3.65 kB
dotcom-rendering/dist/MapEmbedBlockComponent-importable.********************.js	5.11 kB
dotcom-rendering/dist/MapEmbedBlockComponent-importable.legacy.********************.js	5.34 kB
dotcom-rendering/dist/MostViewedFooterData-importable.********************.js	7.75 kB
dotcom-rendering/dist/MostViewedFooterData-importable.legacy.********************.js	7.92 kB
dotcom-rendering/dist/MostViewedRightWrapper-importable.********************.js	6.99 kB
dotcom-rendering/dist/MostViewedRightWrapper-importable.legacy.********************.js	4.95 kB
dotcom-rendering/dist/newsletterEmbedIframe.********************.js	931 B
dotcom-rendering/dist/newsletterEmbedIframe.legacy.********************.js	975 B
dotcom-rendering/dist/OnwardsUpper-importable.********************.js	6.85 kB
dotcom-rendering/dist/OnwardsUpper-importable.legacy.********************.js	7.06 kB
dotcom-rendering/dist/ophan.********************.js	7.24 kB
dotcom-rendering/dist/ophan.legacy.********************.js	7.86 kB
dotcom-rendering/dist/PersonalityQuizAtomWrapper-importable.********************.js	271 B
dotcom-rendering/dist/PersonalityQuizAtomWrapper-importable.legacy.********************.js	283 B
dotcom-rendering/dist/ProfileAtomWrapper-importable.********************.js	264 B
dotcom-rendering/dist/ProfileAtomWrapper-importable.legacy.********************.js	274 B
dotcom-rendering/dist/PulsingDot-importable.********************.js	1.66 kB
dotcom-rendering/dist/PulsingDot-importable.legacy.********************.js	1.76 kB
dotcom-rendering/dist/QandaAtomWrapper-importable.********************.js	262 B
dotcom-rendering/dist/QandaAtomWrapper-importable.legacy.********************.js	274 B
dotcom-rendering/dist/ReaderRevenueDev-importable.********************.js	4.58 kB
dotcom-rendering/dist/ReaderRevenueDev-importable.legacy.********************.js	4.59 kB
dotcom-rendering/dist/readerRevenueDevUtils.********************.js	3.88 kB
dotcom-rendering/dist/readerRevenueDevUtils.js	2.33 kB
dotcom-rendering/dist/readerRevenueDevUtils.legacy.********************.js	4.93 kB
dotcom-rendering/dist/ReaderRevenueLinks-importable.********************.js	5.87 kB
dotcom-rendering/dist/ReaderRevenueLinks-importable.legacy.********************.js	3.34 kB
dotcom-rendering/dist/relativeTime.********************.js	1.3 kB
dotcom-rendering/dist/relativeTime.legacy.********************.js	1.35 kB
dotcom-rendering/dist/RichLinkComponent-importable.********************.js	5.74 kB
dotcom-rendering/dist/RichLinkComponent-importable.legacy.********************.js	5.86 kB
dotcom-rendering/dist/SecureSignupIframe-importable.********************.js	8.43 kB
dotcom-rendering/dist/SecureSignupIframe-importable.legacy.********************.js	7.93 kB
dotcom-rendering/dist/sentry.********************.js	681 B
dotcom-rendering/dist/sentry.legacy.********************.js	688 B
dotcom-rendering/dist/sentryLoader.********************.js	10.1 kB
dotcom-rendering/dist/sentryLoader.legacy.********************.js	14.2 kB
dotcom-rendering/dist/SetABTests-importable.********************.js	4.22 kB
dotcom-rendering/dist/SetABTests-importable.legacy.********************.js	1.38 kB
dotcom-rendering/dist/ShareCount-importable.********************.js	3.59 kB
dotcom-rendering/dist/ShareCount-importable.legacy.********************.js	3.7 kB
dotcom-rendering/dist/shimport.********************.js	2.78 kB
dotcom-rendering/dist/shimport.legacy.********************.js	2.79 kB
dotcom-rendering/dist/ShowHideContainers-importable.********************.js	726 B
dotcom-rendering/dist/ShowHideContainers-importable.legacy.********************.js	761 B
dotcom-rendering/dist/SignInGateMain.********************.js	4.48 kB
dotcom-rendering/dist/SignInGateMain.js	2.87 kB
dotcom-rendering/dist/SignInGateMain.legacy.********************.js	4.56 kB
dotcom-rendering/dist/SignInGateSelector-importable.********************.js	4.97 kB
dotcom-rendering/dist/SignInGateSelector-importable.legacy.********************.js	3.32 kB
dotcom-rendering/dist/SlotBodyEnd-importable.********************.js	2.31 kB
dotcom-rendering/dist/SlotBodyEnd-importable.legacy.********************.js	2.94 kB
dotcom-rendering/dist/SpotifyBlockComponent-importable.********************.js	5.04 kB
dotcom-rendering/dist/SpotifyBlockComponent-importable.legacy.********************.js	5.26 kB
dotcom-rendering/dist/StickyBottomBanner-importable.********************.js	5.08 kB
dotcom-rendering/dist/StickyBottomBanner-importable.legacy.********************.js	5.97 kB
dotcom-rendering/dist/SubNav-importable.********************.js	3.26 kB
dotcom-rendering/dist/SubNav-importable.legacy.********************.js	3.38 kB
dotcom-rendering/dist/TimelineAtomWrapper-importable.********************.js	264 B
dotcom-rendering/dist/TimelineAtomWrapper-importable.legacy.********************.js	275 B
dotcom-rendering/dist/TopicFilterBank-importable.********************.js	3.73 kB
dotcom-rendering/dist/TopicFilterBank-importable.legacy.********************.js	3.84 kB
dotcom-rendering/dist/TopRightAdSlot-importable.********************.js	3.52 kB
dotcom-rendering/dist/TopRightAdSlot-importable.legacy.********************.js	3.56 kB
dotcom-rendering/dist/TweetBlockComponent-importable.********************.js	1.8 kB
dotcom-rendering/dist/TweetBlockComponent-importable.legacy.********************.js	1.79 kB
dotcom-rendering/dist/UnsafeEmbedBlockComponent-importable.********************.js	3.01 kB
dotcom-rendering/dist/UnsafeEmbedBlockComponent-importable.legacy.********************.js	3.11 kB
dotcom-rendering/dist/VideoFacebookBlockComponent-importable.********************.js	5.13 kB
dotcom-rendering/dist/VideoFacebookBlockComponent-importable.legacy.********************.js	5.35 kB
dotcom-rendering/dist/VineBlockComponent-importable.********************.js	2.81 kB
dotcom-rendering/dist/VineBlockComponent-importable.legacy.********************.js	2.91 kB
dotcom-rendering/dist/YoutubeBlockComponent-importable.********************.js	6.36 kB
dotcom-rendering/dist/YoutubeBlockComponent-importable.legacy.********************.js	6.65 kB

_{compressed-size-action}

[jamesgorrie]

Part of #5940