Storybook: Try adding support for `@storybook/test-runner`
Proposed changes:
Storybook has a @storybook/test-runner package, which can be used to make sure the stories at least work (and can be used for integration testing too if such tests are written). This tries adding that package and running it in CI.
To not have to build all the deps before we can run the test, let's set jetpack:src in storybook's webpack config like we do in our own. Hopefully that also means we can remove the extra.dependencies.build from composer.json too, which may speed up the builds a little by not having to have storybook wait until the end.
To make the output from that a bit more friendly, let's also make storybook development builds not minify.
Other information:
- [ ] Have you written new tests for your changes, if applicable?
- [ ] Have you checked the E2E test CI results, and verified that your changes do not break them?
- [ ] Have you tested your changes on WordPress.com, if applicable (if so, you'll see a generated comment below with a script to run)?
Jetpack product discussion
p1708449385672469/1708444782.597729-slack-CDLH4C1UZ Announcement: pdWQjU-Ip-p2
Does this pull request change what data or activity we track or use?
No
Testing instructions:
- Is CI running the storybook test thing?
- Does the output from all the failing tests look sensible?
Thank you for your PR!
When contributing to Jetpack, we have a few suggestions that can help us test and review your patch:
- :white_check_mark: Include a description of your PR changes.
- :white_check_mark: Add a "[Status]" label (In Progress, Needs Team Review, ...).
- :white_check_mark: Add testing instructions.
- :white_check_mark: Specify whether this PR includes any changes to data or privacy.
- :white_check_mark: Add changelog entries to affected projects
This comment will be updated as you work on your PR and make changes. If you think that some of those checks are not needed for your PR, please explain why you think so. Thanks for cooperation :robot:
The e2e test report can be found here. Please note that it can take a few minutes after the e2e tests checks are complete for the report to be available.
Once your PR is ready for review, check one last time that all required checks appearing at the bottom of this PR are passing or skipped. Then, add the "[Status] Needs Team Review" label and ask someone from your team review the code. Once reviewed, it can then be merged. If you need an extra review from someone familiar with the codebase, you can update the labels from "[Status] Needs Team Review" to "[Status] Needs Review", and in that case Jetpack Approvers will do a final review of your PR.
Are you an Automattician? Please test your changes on all WordPress.com environments to help mitigate accidental explosions.
-
To test on WoA, go to the Plugins menu on a WordPress.com Simple site. Click on the "Upload" button and follow the upgrade flow to be able to upload, install, and activate the Jetpack Beta plugin. Once the plugin is active, go to Jetpack > Jetpack Beta, select your plugin, and enable the
try/storybook-testsbranch.- For
jetpack-mu-wpcomchanges, also adddefine( 'JETPACK_MU_WPCOM_LOAD_VIA_BETA_PLUGIN', true );to yourwp-config.phpfile.
- For
-
To test on Simple, run the following command on your sandbox:
bin/jetpack-downloader test jetpack try/storybook-testsbin/jetpack-downloader test jetpack-mu-wpcom-plugin try/storybook-tests
Interested in more tips and information?
- In your local development environment, use the
jetpack rsynccommand to sync your changes to a WoA dev blog. - Read more about our development workflow here: PCYsg-eg0-p2
- Figure out when your changes will be shipped to customers here: PCYsg-eg5-p2
Note we shouldn't merge this until either the tests pass or we figure out the right setting (this I think?) to not email on every PR about the Tests workflow failing because of this.
Looks like that worked.
Also I drafted an announcement for it: pdWQjU-Ip-p2
Looks good to go. P2 post looks good as well. Only blocking thing right now is a merge conflict.
PHP tests: PHP 8.2 WP trunk fail is unrelated, see p1709562619872529/1709562558.619689-slack-C034JEXD1RD for details.