Enable multi-process commit mining in HTTP service
Since bugbug is no longer hosted on Heroku, memory constraints that required single-process commit mining are no longer relevant. This restores parallel processing for performance.
Changes
- Remove
use_single_process=Truefromrepository.download_commits()calls in:-
http_service/bugbug_http/boot.py- commit DB updates during worker boot -
http_service/bugbug_http/models.py- patch analysis for test selection
-
The function defaults to use_single_process=False, which enables ProcessPoolExecutor for parallel commit and patch processing across all available CPU cores.
Original prompt
This section details on the original issue you should resolve
<issue_title>Use multiple processes for mining commits in the HTTP service</issue_title> <issue_description>Since we will no longer host bugbug on Heroku, we are not as concerned with memory usage as before.
Basically reverting b74bf96400d0e845705ae41ec308119125fa31ab and 2ac0bc4c12bf0bd37abcbd631ef4f503a46aeee5 will do the trick.</issue_description>
<agent_instructions>Follow the comments and issues mentioned. </agent_instructions>
Comments on the Issue (you are @copilot in this section)
- Fixes mozilla/bugbug#5387
✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.
Let's wait a bit on this until the new deployment is fully functional.