[CI/Build] migrate static project metadata from setup.py to pyproject.toml

[dtrifiro]

Static metadata can be moved to pyproject.toml and removed from setup.py

[github-actions[bot]]

👋 Hi! Thank you for contributing to the vLLM project. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

• Add ready label to the PR
• Enable auto-merge.

🚀

[youkaichao]

how does this affect the build process of ci, which uses python setup.py ?

[dtrifiro]

@youkaichao It should not. This is just moving static metadata from setup.py to pyproject.toml

Building wheels from this branch and main and looking at metadata doesn't show any difference:

gh pr checkout 8772
VLLM_TARGET_DEVICE=empty python setup.py bdist_wheel
# ^ creates `vllm-0.6.3.dev157+ge3bef869f.empty-py3-none-any.whl`
git checkout main
VLLM_TARGET_DEVICE=empty python setup.py bdist_wheel
# ^ creates `vllm-0.6.3.dev156+g07c11cf4d.empty-py3-none-any.whl`

# you can verify the wheel's generated metadata files:
ls **/*dist-info

Outputs:

vllm-0.6.3.dev156+g07c11cf4d.empty-py3-none-any/vllm-0.6.3.dev156+g07c11cf4d.empty.dist-info: entry_points.txt LICENSE METADATA RECORD top_level.txt WHEEL

vllm-0.6.3.dev157+ge3bef869f.empty-py3-none-any/vllm-0.6.3.dev157+ge3bef869f.empty.dist-info: entry_points.txt LICENSE METADATA RECORD top_level.txt WHEEL

You can compare files, and they're identical (except for the extra embedded license in METADATA in the dev157 build, which is this one)

[hmellor]

@dtrifiro do you plan to update this PR? It seems like a nice step towards removing setup.py

[dtrifiro]

@hmellor we will not be able to remove setup.py as we heavily rely on a custom build process. This PR aimed to move static metadata to pyproject.toml rather then remove setup.py entirely

[dtrifiro]

@simon-mo done.

Some requirements (optional requirements groups) are currently static but I left them in pyproject.yoml regardless for consistency, maybe this could be worth revisiting after https://github.com/vllm-project/vllm/pull/7525 is merged

[dtrifiro]

Build failures look unrelated

[dtrifiro]

@simon-mo can we merge this?

[mergify[bot]]

This pull request has merge conflicts that must be resolved before it can be merged. Please rebase the PR, @dtrifiro.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[NickLucche]

setup.py starting to look a bit cleaner already, thanks!

[dtrifiro]

@simon-mo sorry to ping you again, but could we merge this?

[youkaichao]

QQ: when we use python setup.py , will pyproject.toml still be read?

I'm thinking about this line:

https://github.com/vllm-project/vllm/blob/a02c86b4dd00f096b33499657f2d119be91cc9a6/Dockerfile#L115

[youkaichao]

cc @dtrifiro

[dtrifiro]

@youkaichao yes.