Optimize build commands & building the plugin js

[d-claassen]

Context

• Introduces build responsibility (and Yarn commands) for all separate packages in the monorepo.
• Makes the building of the packages & the plugin js use the built versions of the other monorepo packages (via the main key in each package.json).
• Builds the plugin js in a hierarchical order, where the lowest packages in the monorepo chain are build first when running $ yarn build and packages on the same hierarchical level are built in parallel.
• Introduces better reliance on the NODE_ENV variable for production/development builds.
• Follow-up: Move build tools for the plugin itself (packages/js) from the root package.json to its own package.json, including other files specifically for building the plugin that now live in the root.
• Follow-up: Optimize building of all projects with a build-cache, where unchanged packages don't need a re-build.

Summary

This PR can be summarized in the following changelog entry:

• Optimizes development processes for building JavaScript.

Relevant technical choices:

• Replaced the Grunt alias shell:build-ui-library with generic shell:build-packages, powered by Lerna.
• Introduced separate build commands for all packages.
  • Introduced or improved PostCSS and Babel configuration for building of individual packages.
  • Introduced or improved ESLint and Jest configuration to support (usage of) built packages in the monorepo.
• Made the $ yarn build command in the root also build all packages, with the help of Lerna.
• Updated grunt-eslint in yoastseo to align the included ESLint version with other packages (now all on v7.23.0)
• Updates Jest and @wordpress/scripts to align the included Jest version across packages (now all on v27.5.1)
• Updates the CI pipeline so the JS tests build the packages before running the tests: these tests now use the built version of the monorepo packages.
• Updates existing snapshots:
  • Packages are no longer applying styled-components or other packages, and are therefore ending up with different class names.
  • Updated version of Jest and ESLint come with new errors or stricter assertions.
• Adds to the Jest config a yoastseo specific module name mapper to ensure import .. from "yoastseo" in the package itself are resolved to the src files, and not to the build files.
• I've disabled the tests for the UI library. There's a whole story behind it, but the current tool will be deprecated in v7.6 and fully replaced in v8. In the mean time, it's bringing old sub-dependencies complicating the state of our monorepo's node_modules structure. Since we're investigating updates to the Storybook, I've decided to jump ahead of the future deprecation.

Test instructions

Test instructions for the acceptance test before the PR gets merged

This PR can be acceptance tested by following these steps:

• Check that CI continues to succeed
• Run $ yarn lint in the root of the project.
• Run $ yarn build in the root of the project.
• Run $ yarn test in the root of the project.
• Run $ yarn workspace @yoast/ui-library storybook in the root of the project.
• Run $ yarn why jest to see a single installment of Jest in the node_modules, preventing conflicts throughout the monorepo.
• Smoke test of the full sidebar/metabox, and settings pages.

Relevant test scenarios

• [x] Changes should be tested with the browser console open
• [x] Changes should be tested on different posts/pages/taxonomies/custom post types/custom taxonomies
• [x] Changes should be tested on different editors (Block/Classic/Elementor/other)
• [x] Changes should be tested on different browsers
• [ ] Changes should be tested on multisite

Test instructions for QA when the code is in the RC

• [x] QA should use the same steps as above.

Impact check

This PR affects the following parts of the plugin, which may require extra testing:

• All the JS, but it should not be none of the JS.

UI changes

• [ ] This PR changes the UI in the plugin. I have added the 'UI change' label to this PR.

Other environments

• [x] This PR also affects Shopify. I have added a changelog entry starting with [shopify-seo], added test instructions for Shopify and attached the Shopify label to this PR.

Documentation

• [ ] I have written documentation for this change.

Quality assurance

• [ ] I have tested this code to the best of my abilities.
• [ ] During testing, I had activated all plugins that Yoast SEO provides integrations for.
• [ ] I have added unit tests to verify the code works as intended.
• [ ] If any part of the code is behind a feature flag, my test instructions also cover cases where the feature flag is switched off.
• [x] I have written this PR in accordance with my team's definition of done.
• [x] I have checked that the base branch is correctly set.

Innovation

• [ ] No innovation project is applicable for this PR.
• [x] This PR falls under an innovation project. I have attached the innovation label.
• [ ] I have added my hours to the WBSO document.

Fixes #

[github-actions[bot]]

@d-claassen Please be aware that following packages have been abandoned and are not actively maintained anymore:

Package name	Path
yoast-components	packages/yoast-components
@yoast/babel-preset	packages/babel-preset
@yoast/components	packages/components
@yoast/e2e-tests	packages/e2e-tests
@yoast/helpers	packages/helpers
@yoast/jest-preset	packages/jest-preset
@yoast/style-guide	packages/style-guide

Please consider using the other packages instead.

[coveralls]

Pull Request Test Coverage Report for Build bbc42abf07c4b5d013194c0c400ca96d0b9a4946

Details

• 1 of 1 (100.0%) changed or added relevant line in 1 file are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.07%) to 53.244%

---

Totals
Change from base Build 14a551f319c28c9acdff064bc91beec9741b1b8a:	0.07%
Covered Lines:	29059
Relevant Lines:	54895

---

💛 - Coveralls

[d-claassen]

After an extensive CR session with @igorschoester, we found these steps still need to be taken:

• [x] Don't bluntly ignore unresolved imports for all @yoast/* packages, but specify the packages from the monorepo explicitly.
• [x] Add dependencies on new tools in the package.json file for each package. So add Babel, presets, plugins, etc.
• [x] Split up the yarn build script of the yoastseo package into separate smaller scripts