Tianqi Chen
Tianqi Chen
seems you are not installing into the conda env, and python3 you invoked was not from conda env. try to run `which python3` to ensure the python comes from conda,...
usually conda install python would suffice, then you run `python -m pip install` instead of `python3 -m pip` which goes into your system python
what i meant was run `python -m pip install --pre -U -f https://mlc.ai/wheels mlc-llm-nightly mlc-ai-nightly` Make sure you also run `conda install python pip`
you do need to install the necessary dependencies such as numpy, usually they should come with pip install, but given it is missing, you need to manually install it
unfortunately seems your python did not pick it up, i dn't know why, seems to be issue with your loal env
I am sorry i don't really know, normally if the conda environment contains the python and you dpn't use your system python, and that python contains all necessary dependency, we...
likely we need to build without flashinfer (as it does not support pre-SM80) @Hzfengsy have some followup replacement that might help
For now, we might need to build mlc-llm from source by setting flashinfer OFF https://llm.mlc.ai/docs/install/mlc_llm.html#option-2-build-from-source
ah yes we also need build tvm from source, since the flashinfer was packaged through runtime there
I think we could update engine to have a runtime compatibility check, so we can leverage flashinfer wheel. However, looking a bit more, because there are so many compute compact...