[Release] Unify local build scripts to use `cibuildwheel` and reduce size of sdist
- sdist: Update sdist includes, reduce size to meet the 100m limit of pypi;
- GA: List artifacts for pull requests with title
"[Release]", then we can download built wheel/sdist from artifacts, instead of use qemu on local machine. This also produces macos wheels, example: https://github.com/tile-ai/tilelang/actions/runs/19007774631?pr=1171 @LeiWang1999 ; - sdist: Generates a
.git_commit.txtin sdist.pipdo not allow a sdist to have different version with the installed package (we cannot have a sdist namedtilelang-0.1.6.post2+gitxxxx.tar.gzand install a package namedtilelang==0.1.6.post2+cu128.gitxxxx). Since sdist should not contain toolchain info, we need to store its git commit separately. When installed from sdist, the git info will be read from that file. - scripts: delete some file, use
cibuildwheelto generate wheels for linux. However this seems not necessary, I hope Github Actions can eliminated most of local builds.
cc @XuehaiPan
Summary by CodeRabbit
- Chores
- Removed legacy manifest file and moved to explicit vendored subcomponent packaging in pyproject.
- Simplified build/distribution: removed multi-arch Docker orchestration, using direct build-tool invocations for local/CI builds.
- Updated container builder bases and per-architecture CUDA defaults for manylinux builds.
- Persisted version stamping to a git-pin file (now ignored) and fallback to "gitunknown" when commit info is unavailable.
- CI: sdist/wheels/artifacts also produced for PRs titled “[Release]”.
👋 Hi! Thank you for contributing to the TileLang project.
Please remember to run pre-commit run --all-files in the root directory of the project to ensure your changes are properly linted and formatted. This will help ensure your contribution passes the format check.
We appreciate you taking this step! Our team will review your contribution, and we look forward to your awesome work! 🚀
[!WARNING]
Rate limit exceeded
@oraluben has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 7 minutes and 5 seconds before requesting another review.
⌛ How to resolve this issue?
After the wait time has elapsed, a review can be triggered using the
@coderabbitai reviewcommand as a PR comment. Alternatively, push new commits to this PR.We recommend that you space out your commits to avoid hitting the rate limit.
🚦 How do rate limits work?
CodeRabbit enforces hourly rate limits for each developer per organization.
Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.
Please see our FAQ for further information.
📥 Commits
Reviewing files that changed from the base of the PR and between 703e63c15803fd016f321f0ef6e1ef73eb912156 and 54ff485957e25c2c96f97ada83769935b37bb7e1.
📒 Files selected for processing (1)
pyproject.toml(5 hunks)
Walkthrough
Deletes MANIFEST.in, tightens sdist/wheel vendoring in pyproject.toml, simplifies multi-arch Docker distribution scripts to single cibuildwheel invocations, updates manylinux Dockerfile and CI build flags, adds git commit caching/persistence in version_provider.py, and ignores _git_commit.txt in .gitignore.
Changes
| Cohort / File(s) | Summary |
|---|---|
Packaging manifests MANIFEST.in, pyproject.toml, version_provider.py, .gitignore |
MANIFEST.in removed; pyproject.toml narrows sdist/wheel vendoring to explicit 3rdparty subpaths (TVM, CUTLASS, Composable Kernel), adds DEFAULT_CUDA_VERSION and arch-aware CUDA defaults, and build/debug flags; version_provider.py adds @lru_cache(maxsize=1), persists git hash to _git_commit.txt and reads it as fallback, and appends gitunknown when missing; .gitignore now ignores _git_commit.txt. |
Top-level build orchestration maint/scripts/docker_build_all.sh |
Removed sequential invocation and logging of docker_local_distribute.sh and docker_pypi_distribute.sh. |
Local distribution script maint/scripts/docker_local_distribute.sh |
Replaced multi-arch docker buildx orchestration with a single local cibuildwheel invocation (CIBW_BUILD targeted at local arch). |
PyPI distribution script maint/scripts/docker_pypi_distribute.sh |
Replaced per-arch docker buildx loop and dist relocation with a single cibuildwheel invocation exposing CIBW_ARCHS, CIBW_BUILD, and NO_VERSION_LABEL; removed per-arch tagging/relocation logic. |
Manylinux builder Dockerfile maint/scripts/pypi.manylinux.Dockerfile |
Switched builder bases to quay.io/pypa/manylinux... images, added ARG/ENV CUDA_VERSION and TARGETARCH multi-stage selection, set TZ, updated CUDA repo handling, and install cibuildwheel via pipx (removed venv/uv steps). |
GitHub Actions .github/workflows/dist.yml |
Replaced NO_GIT_VERSION with NO_VERSION_LABEL; broadened conditions so SDist, wheel upload, and listing artifacts also run for PRs whose titles contain [Release]. |
Sequence Diagram(s)
sequenceDiagram
rect rgb(250,240,230)
Note over CI: Old flow — build-all invoked per-arch Docker buildx scripts
CI->>docker_build_all.sh: run
docker_build_all.sh->>docker_local_distribute.sh: invoke & log
docker_local_distribute.sh->>docker buildx: setup & per-arch builds
docker_build_all.sh->>docker_pypi_distribute.sh: invoke & log
docker_pypi_distribute.sh->>docker buildx: setup & per-arch builds
docker buildx-->>CI: per-arch artifacts (relocated)
end
rect rgb(230,250,240)
Note over CI: New flow — direct cibuildwheel invocation
CI->>cibuildwheel: invoke (CIBW_ARCHS/CIBW_BUILD/NO_VERSION_LABEL)
cibuildwheel->>manylinux container: build wheels (arch-aware)
manylinux container-->>cibuildwheel: produced wheels
cibuildwheel-->>CI: artifacts in dist/
end
Estimated code review effort
🎯 3 (Moderate) | ⏱️ ~25 minutes
- Inspect pyproject.toml sdist/wheel include/exclude changes vs. removed MANIFEST.in.
- Verify atomicity and permissions for
_git_commit.txtwrites/reads inversion_provider.py. - Validate manylinux Dockerfile CUDA repo and pipx-based cibuildwheel installation.
Possibly related PRs
- tile-ai/tilelang#973 — modifies CI distribution/workflow logic overlapping
.github/workflows/dist.ymlchanges. - tile-ai/tilelang#1098 — touches packaging and CI manifests (MANIFEST.in, pyproject.toml) similar to these packaging edits.
Suggested reviewers
- LeiWang1999
Poem
🐇 I nudged the manifest into the ground,
Packed TVM and CUTLASS safe and sound.
From buildx bustle to cibuildwheel’s cheer,
I cached the hash so versions persevere.
Hoppity-hop — the dist folder's near! 🥕
Pre-merge checks and finishing touches
✅ Passed checks (2 passed)
| Check name | Status | Explanation |
|---|---|---|
| Description Check | ✅ Passed | Check skipped - CodeRabbit’s high-level summary is enabled. |
| Title check | ✅ Passed | The title accurately summarizes the two main changes: unifying build scripts to use cibuildwheel and reducing sdist size, which directly align with the changeset modifications. |
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Comment @coderabbitai help to get the list of available commands and usage tips.
is 3rdparty/tvm/src necessary for wheels?
is
3rdparty/tvm/srcnecessary for wheels?
yes, this should be included