mlc-ai
which commit of relax should be used ?
python3 build.py --hf-path databricks/dolly-v2-3b --quantization q3f16_0 --max-seq-len 768 Weights exist at dist/models/dolly-v2-3b, skipping download. Using model path dist/models/dolly-v2-3b Automatically configuring target: cuda -keys=cuda,gpu -arch=sm_80 -max_num_threads=1024 -thread_warp_size=32 Segmentation fault (core dumped) 出错的代码是mod = relax.pipeline.get_pipeline()(mod)
I used code from this repo: https://github.com/mlc-ai/relax just the latest commit.
Did you manually set the USE_CUDA when you were building the relax proj?
Did you manually set the USE_CUDA when you were building the relax proj?
yes, I set USE_CUDA ON, and use LLVM，others are default
met the same issue.
The only difference is I manually download dolly-v2-3b from hf and replace _name_or_path: EleutherAI/pythia-2.8b with _name_or_path: databricks/dolly-v2-3b in config.json to pass the check from this function https://github.com/mlc-ai/mlc-llm/blob/main/mlc_llm/utils.py#L50-L76.
Try the CPU version of pytorch. CUDA pytorch was making me segfault for some reason, and isnt needed.
Try the CPU version of pytorch. CUDA pytorch was making me segfault for some reason, and isnt needed.
Hi, I have tried the CPU-only version of Pytorch, but met the same Segmentation fault issue.
I tried the nightly build of TVM Unity by following the instructions here, and it works for me.
pip3 install mlc-ai-nightly -f https://mlc.ai/wheels
I tried the nightly build of TVM Unity by following the instructions here, and it works for me.
pip3 install mlc-ai-nightly -f https://mlc.ai/wheels
works for me, thx ~
Got the same issue, and fixed with change LLVM from 16.0(Newest) to LLVM 15.06(Old version). Thanks,
Please check out this page for proper installation of TVM: https://github.com/mlc-ai/mlc-llm/blob/main/docs/install/tvm.rst
