setup-matlab
setup-matlab copied to clipboard
matlab-actions
Native support for PCT parallel run on self hosted runners in run-tests
Hi, we're an automotive firmware consulting firm.
I'll start off saying: thanks for all your hard work on making it easy to integrate and enable CI with MATLAB. I am pleased to say that all of your actions published so far work great, and integrating everything was an absolute breeze. This has been a very important milestone for MATLAB and all our ISO 26262/high assurance clients, making it feasible to introduce CI pipelines into new projects earlier in the development cycle.
At the moment we have some clients with self-hosted runners and private repositories. These runners have special licensing installed (i.e. high integrity compilers, toolchains, blocksets, etc.)
For some of our clients, the runners are starting to fall behind as more unit tests are added and this causes a back-up in the system during the day that can delay developers. Adding more runners is not a viable strategy at the moment because the licensing and tooling is quite costly as you can imagine, and we're already running on the fastest boxes possible.
When we get to this stage in a project, we usually tool the CI to use PCT (Parallel Toolbox) to run the tests in parallel using a local thread pool. In our experience there is a lot of CPU idle time in test tear-up and tear-down, so this decreases the amount of time taken to run the tests linearly, i.e. if there are 8 workers created on 4 cores, then the tests will run approx. in 1/8th the time (87% time and cost saving!)
For recent clients however, we've stopped rolling our own CI solution using m-files and started using the run-tests action. This is great because it means that we can deliver a CI solution that is much more standard and out-of-the-box, but it also means that we can't just customize the solution to enable PCT anymore, so we're now sort of stuck on the performance side.
It would naturally sell more PCT licenses to us and our clients if run-tests natively detected an installation of PCT on the worker and automatically utilized it, since it would be very convenient to roll out. If PCT can not eventually be supported, it would put us in a position where we might have to use a different solution. I expect many might feel the same.
The ability to leverage parallel pools for self hosted runners was added in v1.2.0 of run-tests 🎉
https://github.com/matlab-actions/run-tests/releases/tag/v1.2.0
