mlc-llm
mlc-llm copied to clipboard
iOS deployment of mlc-llm has compilation problems
By process: 1、Install TVM Unity and compile successfully 2、Get the model weight 3、Build the model to the library exist python3 build.py --model vicuna-v1-7b --type float16 --target iphone --quantization-mode int3 --quantization-sym --quantization-storage-nbit 16 --max-seq-len 768
TVMError: backtracking (last call): File "xxx/mlc-llm/relax/src/support/parallel_for.cc", line 139 RuntimeError: parallel_for_dynamic error with [15:28:54]. xxx/mlc-llm/relax/src/meta_schedule/database/json_database.cc: 202: ValueError: Unable to parse TuningRecord , on Line 23 of file log_db/vicuna-v1-7b/database_tuning_record.json. The workload is: from tvm.script import ir as I from tvm.script import tir as T
Which step went wrong: TVM installation or getting the model?