dd-trace-py icon indicating copy to clipboard operation
dd-trace-py copied to clipboard

chore(ci): parallelize build wheels

Open taegyunkim opened this issue 7 months ago • 2 comments

Parallelize building wheel using unique identifier used by cibuildwheel, which is a combination of python version, target OS, and target architecture.

We've been building wheels in parallel across OS's used for builds, as defined here. For ubuntu-latest, it has to build all combinations of [manylinux, musllinux] * [x86_64, i686] in serial, which takes about ~23 mins.

However, when split into unique identifier, the slowest one takes about ~9 mins.

To reviewer: We'd need to change the set of required CI checks from previous set of build_wheels to these new ones. Please let me know how I can do this.

Checklist

  • [x] The PR description includes an overview of the change
  • [x] The PR description articulates the motivation for the change
  • [x] The change includes tests OR the PR description describes a testing strategy
  • [x] The PR description notes risks associated with the change, if any
  • [x] Newly-added code is easy to change
  • [x] The change follows the library release note guidelines
  • [x] The change includes or references documentation updates if necessary
  • [x] Backport labels are set (if applicable)

Reviewer Checklist

  • [ ] Title is accurate
  • [ ] All changes are related to the pull request's stated goal
  • [ ] Avoids breaking API changes
  • [ ] Testing strategy adequately addresses listed risks
  • [ ] Newly-added code is easy to change
  • [ ] Release note makes sense to a user of the library
  • [ ] If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
  • [ ] Backport labels are set in a manner that is consistent with the release branch maintenance policy

taegyunkim avatar Jul 03 '24 19:07 taegyunkim