[CI/Build] Platform agnostic wheel

[tomeras91]

This PR enables build of a platform-agnostic wheel which is installable also on macos. The idea is to improve the dev-experience for creating projects that import and use vLLM. Important: This wheel does not enable running of vllm on mac, but does allow to import it.

The PR doesn't entirely fix issues #212, #695, #1397, #1921, but it's a step forward.

[simon-mo]

Interestingly Triton recently supported macOS with Apple Silicon: https://github.com/triton-lang/triton/issues/3443

[tomeras91]

Interestingly Triton recently supported macOS with Apple Silicon: triton-lang/triton#3443

Yep. But AFAIU they still release wheels only for Linux so it can't be pip installed directly from pypi. I guess building from source is the only option to get triton installed on mac. https://pypi.org/project/triton/2.3.0/#files

[hugolytics]

this would be great! maybe a warning at import could be a good idea, our team develops on mac, and we run the LLM workloads on a server. But it would be great to be able to install the project locally for debugging some of the helper code without having to maintain separate deps.

[cdpierse]

➕ 1 for this feature. It would be really nice to enable local development for code completion, and documentation references.

[youkaichao]

I think this feature is useful, but can you use the existing infra without introducing another env var?

e.g. VLLM_TARGET_DEVICE="empty" python setup.py develop

[tomeras91]

I think this feature is useful, but can you use the existing infra without introducing another env var?

e.g. VLLM_TARGET_DEVICE="empty" python setup.py develop

Thanks @youkaichao . Done now also - the diff between this an main is very minimal due to the great work in https://github.com/vllm-project/vllm/pull/6786, solving the import issues with triton

[tomeras91]

@youkaichao - now this PR actually does 2 things:

1. It enables a local platform agnostic build of vLLM with VLLM_TARGET_DEVICE="empty" python setup.py
2. It actually allows build of the published tar.gz on macs. So now a developer working on mac can run pip install vllm and actually have vllm installed and importable (though not runnable)

IMO the second point is the nicer feature here. Really makes development using vLLM much easier on macs. I'm not sure though if you'd want to include it.. so if you think it's not a good idea, we can discard it and just leave the VLLM_TARGET_DEVICE="empty" option

[youkaichao]

@tomeras91 this feature is only for dev and debugging. I don't think it make any sense to publish to pypi. Making VLLM_TARGET_DEVICE="empty" python setup.py work is enough.

[tomeras91]

@tomeras91 this feature is only for dev and debugging. I don't think it make any sense to publish to pypi. Making VLLM_TARGET_DEVICE="empty" python setup.py work is enough.

I understand and agree. I wasn't suggesting to publish the "empty" wheel to pypi. The change here just makes it possible to install the already published tar.gz on mac

[youkaichao]

@tomeras91 can you add a followup pr for https://docs.vllm.ai/en/latest/getting_started/installation.html , to tell users how to use this feature?

[tomeras91]

@tomeras91 can you add a followup pr for https://docs.vllm.ai/en/latest/getting_started/installation.html , to tell users how to use this feature?

Sure - here: https://github.com/vllm-project/vllm/pull/7403