wp-calypso
wp-calypso copied to clipboard
E2E: update FSE: Smoke Test spec to take into account non-iframed editor.
Proposed Changes
This PR updates the FSE: Smoke Test spec to account for the scenario where the non-iframed editor is loaded (aka the Atomic editor).
Key changes:
- if/else statement in the last step
Editor canvas loads
that branches depending on the URL loaded.
Context: We have two types of block editors: iframed and non-iframed. The iframed editor is typically loaded on Simple sites. The non-iframed editor is usually seen on Atomic sites.
It appears that due to some issues (likely related to third-party cookies) sometimes the full site editor loads with the non-iframed editor (which is the fallback), and this causes the original testes to fail since it only accounted for the iframed editor.
Testing Instructions
Ensure the following build configurations are passing:
- [x] Gutenberg E2E (desktop)
- [x] Gutenberg E2E (mobile)
- [x] Calypso E2E (desktop)
- [x] Calypso E2E (mobile)
Pre-merge Checklist
- [ ] Have you written new tests for your changes?
- [ ] Have you tested the feature in Simple (P9HQHe-k8-p2), Atomic (P9HQHe-jW-p2), and self-hosted Jetpack sites (PCYsg-g6b-p2)?
- [ ] Have you checked for TypeScript, React or other console errors?
- [ ] Have you used memoizing on expensive computations? More info in Memoizing with create-selector and Using memoizing selectors and Our Approach to Data
- [ ] Have we added the "[Status] String Freeze" label as soon as any new strings were ready for translation (p4TIVU-5Jq-p2)?
Fixes https://github.com/Automattic/wp-calypso/issues/68059.
This PR does not affect the size of JS and CSS bundles shipped to the user's browser.
Generated by performance advisor bot at iscalypsofastyet.com.
@john-legg I've assigned you this review. If you aren't sure, feel free to reach out to me or hop on a quick call for me to explain the changes.
The failure in Gutenberg simple E2E tests edge (mobile) is due to the build being manually added then cancelled by me. It was an accidental build trigger.
Sorry for the delay, @worldomonation. The changes make sense to me! I'll go ahead and approve it.
question (non-blocking): Do you mind showing me the sites you used to test the non-iframe vs iframe editor sometime? I have very few test sites as a Newmattician, so I'm just curious :)
Yeah, that's a good question!
This is how I went ahead with the investigation and verified that this solution should work for both iframed and non-iframed editors:
- download the trace files from a failed run.
- inspect the captured DOM at the time of failure, where I noticed that the editor was not iframed.
- check the URL that was captured in the trace file.
At this point, I put the two together and figured out it was likely due to some issues with loading the iframed editor (which is used on Simple sites), resulting in the non-iframed editor being loaded as fallback.
From that point on:
- copy the URL for the non-iframed editor and force a personal test site's FSE editor to load in the non-iframed editor.
- make changes to the script.
- instead of navigating using the Sidebar, use
page.goto
to target the non-iframed editor to make sure the test passes.