citgm
citgm copied to clipboard
nodejs
Suggestion: `citgm-smoker-lite` job for PRs
This was briefly discussed in a Release WG meeting. A similar concept to https://ci.nodejs.org/view/All/job/node-test-commit-lite/, we could add a CITGM 'lite' job that only covers one or two most critical platforms (probably would make sense to be rhel8-x64 and osx11).
Running across all platforms takes a lot of time to both run, and interpret triage the results. Having a lite job running means we could increase our usage of CITGM on PRs. The more PRs we run it on, the earlier and easier it is to catch regressions, and may be easier for the wider collaborator base to keep up with the state.
(Perhaps eventually even a citgm-smoker-lite-daily job could make sense to cover the state over time - if there are volunteers to look at it.)
Releasers would still run CITGM and triage on the full set of platforms as part of the release process, but in theory they'd have an easier job as a number of issues are likely to have been caught by the lite job.
Happy to volunteer to set this job up if I can be granted access to do so. It doesn't seem like it'd require much more than duplicating the job and removing the other platforms.
Would you believe it should be a default workflow for all PRs or an opt-in such as request-ci label? I think the latter is reasonable.
@BethGriggs I created https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker-lite/ as a copy of citgm-smoker and gave you permissions to configure it.
Would you believe it should be a default workflow for all PRs or an opt-in such as request-ci label? I think the latter is reasonable.
Probably makes sense to start with opt-in until we see how often it's used and how often it stays passing. Are you thinking of a separate request-citgm label? I'd be +1 to that.
Are you thinking of a separate request-citgm label?
Yes
Why not use GitHub Actions instead of a Jenkins setup as it should only run on Linux, OSX, and Windows?
Why not use GitHub Actions instead of a Jenkins setup as it should only run on Linux, OSX, and Windows?
I feel there's probably some value to running the CITGM lite job using the same environments and machines that we use to run CITGM during releases. Otherwise we (releasers) may spend time debugging failures between the two environments - perhaps negating some of the benefit of running the lite job? I can't really quantify how frequently that would happen, though. But, I do know we've seen CITGM runs impacted by environment/machine in the past.
I also find the per platform per module breakdown of test results in https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/3342/testReport/ quite useful - but maybe we could replicate that easily in actions using --markdown output option or similar.