[rush-lib] Added Build Time and Simulate analysis on the enhanced telemetry files from #3596

[hanksterhan]

Usage: $ rush analyze-telemetry --build-times --file telemetry_datestamp.json $ rush analyze-telemetry -b --file telemetry_datestamp.json $ rush analyze-telemetry --simulate --file telemetry_datastamp.json

Summary

When building projects, it is very helpful to have telemetry data to be able to look inside a build. I have enriched the logged telemetry data by adding the dependency graph, the individual project build time+status, as well as the build machine metrics e.g. number of CPUs, computer architecture, etc. In addition, I wrote a new Rush Action called AnalyzeTelemetryAction that contains two functions (analysis and simulate) that will process this enriched telemetry data to provide some useful dependency analysis. --build-times or -b will identify the projects which are chokepoints during the build. Reducing the build times of these projects will directly reduce the overall build time by 1 second. --simulate will identify the optimal number of CPU cores to build your project.

• Both the --build-times and --simulate flags require an additional --file flag that will point to a telemetry file in the telemetry folder.

How it was tested

Invoked rush analyze-telemetry --build-times --file telemetry_datestamp.json and rush analyze-telemetry --simulate --file telemetry_datestamp.json and the printed output was formatted correctly. The telemetry file was found to be enriched after runs of rush build or rush rebuild.

I implemented quite a few changes but I am very open to refactoring suggestions as this is not my code base. I did my best to limit the number of new imports and to reuse existing code. I apologize in advance that the initial code is in 1 commit - I started writing this code in a way that was not visible to the rushstack repository and had to transfer my work to a forked repository.I look forward to collaborating on this PR, thanks!

[ghost]

All CLA requirements met.

[hanksterhan]

• New Rush Action rush analyze implemented to be stand alone from rush build
• Implemented rush analyze --dependency-analysis and rush analyze --simulate
• makeDependencyGraph creates the dependencyGraph from the results as opposed to creating it from the RushConfiguration.
• Refactored dependents to consumers
• Added build machine information to the logged telemetry e.g. number of CPU cores, machine architecture type, etc.

[dmichon-msft]

I'm going to create a separate PR to add the requested telemetry data to the default telemetry files. I believe it would be cleanest to then convert the analyzer to a standalone CLI tool (i.e. its own package, not part of rush-lib) that can operate on a telemetry file in the standard format. This will extend the utility of the feature to also work with the new multi-phase Heft work that is currently ongoing, allow faster iteration on the tool itself, and not bind it to the guts of rush-lib.

[octogonz]

I'm going to create a separate PR to add the requested telemetry data to the default telemetry files. I believe it would be cleanest to then convert the analyzer to a standalone CLI tool (i.e. its own package, not part of rush-lib) that can operate on a telemetry file in the standard format. This will extend the utility of the feature to also work with the new multi-phase Heft work that is currently ongoing, allow faster iteration on the tool itself, and not bind it to the guts of rush-lib.

I agree that this will reduce the amount of API reviews that would slow down this feature.

That said, I think it's a cool feature for Rush that we would be interested to showcase and help maintain.

• If you want to create a separate CLI tool, we could still host it in the https://github.com/microsoft/rushstack repo (e.g. under the apps/analyze-telemetry folder, maybe published as @rushstack/analyze-telemetry?).

• Another idea would be to keep it integrated with Rush as a rush analyze-telemetry command, but implement it as a plugin package. There's a gallery of example projects here, and a CLI example here: rush-sort-package-json/command-line.json. If you go this route, then your project folder could be something like rush-plugins/rush-analyze-telemetry-plugin.

@hanksterhan BTW sorry if there seemed to be a fire hose of feedback for this PR. The ideal flow is something like:

prototype --> design discussion --> design note --> pull request --> code review

For open source projects, these steps often happen in a different order. 😄

[hanksterhan]

@octogonz @dmichon-msft Thank you for all the feedback. I will work on the changes but I will wait to lean on the telemetry files created as part of #3596

[hanksterhan]

I have refactored all the analysis to be done inside of AnalyzeTelemetryAction instead of in BuildTimePlugin, DependencyAnalysisPlugin, and PhasedActionScript. The next step is to follow octogonz's direction and implement rush analyze-telemetry as a plugin package