ReportGenerator icon indicating copy to clipboard operation
ReportGenerator copied to clipboard

Parallelise Report Generator

Open afscrome opened this issue 5 months ago • 0 comments

I've had a go at trying to parallelise the report generation - aimed particularly at helping GitHub Actions / Azure DevOps where the report generator can be quite slow, presumably due to disk IO. I've read some issues about previous attempts and realised it's not as simple as processing classes in parallel as not all the IReportBuilder implements support concurrency.

With this in mind, I believe two parts of the report generation process (ReportGenerator) can be parallelised

  1. The initial File Analysis can be done concurrently with the report generation
  2. Introduced IParallelisableReportBuilder - IReportBuilders that also implement IParallelisableReportBuilder can then be processed in parallel

this PR has a draft implementation of this, and with the change, I have been able to get report generation (measuring time spent in Generator.GenerateReport) down from 21 secs to down as low as 4 seconds with extreme concurrency, and 6secs with more reasonable concurrency levels.

This PR is still a bit of a work in progress - in particular I need to plumb the concurrency level through as a config option, and I also want to do some testing with GitHub Actions / Azure Devops. However I wanted to raise it in it's current form for some initial thoughts on the approach.

Note, this parallelisation change highly benefits form #695

afscrome avatar Sep 14 '24 11:09 afscrome