blt
blt copied to clipboard
Improve the multi-site automated testing and build process
The provided automated build scripts work well when testing a single site. If you have the need to test multiple variations of a site, in the case of something like ACSF, the scripts do not suit that. As a quick example, code validation is run for each variant instead of a single time. This can add minutes per variant.
It would be nice if the provided build scripts were refactored to be more mindful of duplicated checks and unneeded dependency installs.
I have a PR available that integrates several changes I've made in a current ACSF project where testing variant sites from an ACSF project is a requirement. These changes save roughly 20 minutes per successful build for us, and significantly more when a build fails early. The main summary is:
- Add more build stages and pull some steps out into those
- Only sync if all previous build stages pass
- Only install chromedriver if we need it for the current stage
I do not expect this PR to be merged as-is (or at all). I wanted to provide a summary of the changes I made so that others might have a reference point if they have a similar use-case. I do think some of the changes made here could be integrated into the project without impacting non-multisite/variant builds.
Actually, I am not going to create a PR unless asked otherwise. I forgot that GitHub will show commits from forked repos inside the issue.
Thanks, I think people will find this useful as a template. As you observed, the challenge with getting this into BLT core is ensuring that it works for all use cases, and especially doesn't impact existing single-site users, which are the majority of the user base.
I think at least breaking the travis/pipelines builds into "validate" and "test" stages could make sense, with comments indicating how to run tests for multiple sites.