bugbug icon indicating copy to clipboard operation
bugbug copied to clipboard

Enable multi-process commit mining in HTTP service

Open Copilot opened this issue 5 months ago • 1 comments

Since bugbug is no longer hosted on Heroku, memory constraints that required single-process commit mining are no longer relevant. This restores parallel processing for performance.

Changes

  • Remove use_single_process=True from repository.download_commits() calls in:
    • http_service/bugbug_http/boot.py - commit DB updates during worker boot
    • http_service/bugbug_http/models.py - patch analysis for test selection

The function defaults to use_single_process=False, which enables ProcessPoolExecutor for parallel commit and patch processing across all available CPU cores.

Original prompt

This section details on the original issue you should resolve

<issue_title>Use multiple processes for mining commits in the HTTP service</issue_title> <issue_description>Since we will no longer host bugbug on Heroku, we are not as concerned with memory usage as before.

Basically reverting b74bf96400d0e845705ae41ec308119125fa31ab and 2ac0bc4c12bf0bd37abcbd631ef4f503a46aeee5 will do the trick.</issue_description>

<agent_instructions>Follow the comments and issues mentioned. </agent_instructions>

Comments on the Issue (you are @copilot in this section)

  • Fixes mozilla/bugbug#5387

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot avatar Oct 29 '25 17:10 Copilot

Let's wait a bit on this until the new deployment is fully functional.

marco-c avatar Oct 29 '25 19:10 marco-c