cargo-hack icon indicating copy to clipboard operation
cargo-hack copied to clipboard

feat(multithread): add rayon & par_iter for multithreaded execution

Open NishantJoshi00 opened this issue 2 years ago • 5 comments

Description

This is a feature-gated implementation of multi-threading for this project. This utilizes rayon for performing multi-threading.

This PR is raised against #128.

Reasoning

Following are some of the decisions taken while building the solution. Feel free to comment on any of them, as I might have missed something during the process.

Reason for using #[cfg(...)]

  • In the current implementation of the code, all the smart pointers used aren't thread-safe. This being the case, thread-safety wasn't a requirement for the implementation as these pointers are much more performant than their thread-safe alternatives Rc<T> -> Arc<T>. While some might require multithreading others might want to go with the legacy implementation and I all the smart pointers are made to be thread-safe this could hamper the performance of the legacy implementation. This is avoided by using #[cfg(... gates on variables and arguments.

Reason for TargetDirPool

  • Adding multi-threading to the entire implementation isn't difficult the main problem is to tackle the locking mechanism that cargo has in place. In order to do so multiple target directories were created and they were assigned for execution, as well as reused to take advantage of caching

Reason for rayon

  • While using the traditional multi-threading model was always into consideration, rayon had some added benefits. Being lightweight and guaranteeing data-race freedom, with the added benefit of having great abstraction.

Ignored tests: 2

  • Here I have ignored 2 test cases as the order of the output was not what was expected in the assertion. I wasn't able to implement tests to replace them. Please, feel free to modify them as necessary

NishantJoshi00 avatar Jan 26 '23 10:01 NishantJoshi00