feat(general): POC - Add fork with Pool
We originally used a Pipe per process, and each process received a fixed batch of files ahead of time. Once a process finished its batch, it wrote the results to its pipe and exited. This caused severe load imbalance: if one process happened to receive heavier files, it worked much longer than the others. On top of that, we had a bug in how we handled the pipes, which caused processes to get stuck before actually finishing.
We switched from the pipe-based approach to a Pool, which internally works like a queue: all files are pushed into a shared task queue, and each worker takes a small batch of files, processes them, and then immediately pulls more. This ensures dynamic load balancing - heavy files are naturally spread across workers, and no worker gets stuck with a disproportionately heavy batch.
With this change, moving from the single worker under the old pipe model to 12 workers using a Pool, we improved total of runtime around 5.9× speed-up in a specific test case of a big repo.