[Bug] free(): invalid pointer

[pipul]

🐛 Bug

To Reproduce

python3 build.py --hf-path databricks/dolly-v2-3b --quantization q3f16_0 --max-seq-len 768 Weights exist at dist/models/dolly-v2-3b, skipping download. Using path "dist/models/dolly-v2-3b" for model "dolly-v2-3b" Database paths: ['log_db/redpajama-3b-q4f16', 'log_db/redpajama-3b-q4f32', 'log_db/dolly-v2-3b'] Failed to detect local GPU, falling back to CPU as a target Target configured: llvm -keys=cpu Load cached module from dist/dolly-v2-3b-q3f16_0/mod_cache_before_build_cpu.pkl and skip tracing. You can use --use-cache=0 to retrace Dump static shape TIR to dist/dolly-v2-3b-q3f16_0/debug/mod_tir_static.py Dump dynamic shape TIR to dist/dolly-v2-3b-q3f16_0/debug/mod_tir_dynamic.py Finish exporting to dist/dolly-v2-3b-q3f16_0/dolly-v2-3b-q3f16_0-cpu.so Finish exporting chat config to dist/dolly-v2-3b-q3f16_0/params/mlc-chat-config.json free(): invalid pointer Aborted

Steps to reproduce the behavior:

Expected behavior

Environment

• Platform (e.g. WebGPU/Vulkan/IOS/Android/CUDA):
• Operating system (e.g. Ubuntu/Windows/MacOS/...): ubuntu
• Device (e.g. iPhone 12 Pro, PC+RTX 3090, ...) x86
• How you installed MLC-LLM (conda, source): source
• How you installed TVM-Unity (pip, source): source
• Python version (e.g. 3.10): 3.10
• GPU driver version (if applicable): no GPU
• CUDA/cuDNN version (if applicable):
• TVM Unity Hash Tag (python -c "import tvm; print('\n'.join(f'{k}: {v}' for k, v in tvm.support.libinfo().items()))", applicable if you compile models):
• Any other relevant information:

Additional context

[yzh119]

Actually, this error message will not influence (though annoying) the model building and you have already finished it.

The reason you get this is that you haven't hidden LLVM symbol when compiling TVM Unity and there is some symbol conflict when a python program imports both tvm and other packages such torch that also depends on LLVM.

You cou can install our prebuilt wheels where we already hide symbols.

If you build from source, you should append these two lines in the config.cmake:

set(USE_LLVM "llvm-config --ignore-libllvm --link-static")
set(HIDE_PRIVATE_SYMBOLS ON)

We have fixed our docstring #275 accordingly.