Tianqi Chen

icon indicating a website link https://tqchen.com/

icon company CMU, NVIDIA icon location Machine Learning and Systems

Results 637 comments of Tianqi Chen

[Bug] Check failed: (args.size() == initial_indices_orig.size()) is false

Thanks for reporting if it is possibe to get a minimum repro that would be helpful. You can do so by dumping out the TVMScript before the transform, minimize it...

/opt/AI/llm_obj/mlc-llm/3rdparty/tvm/jvm/native/src/main/native/org_apache_tvm_native_c_api.cc:232:31: error: cannot initialize a parameter of type 'void **' with an rvalue of type 'JNIEnv **' (aka 'JNIEnv_ **') 232 | _jvm->AttachCurrentThread(&env, nullptr); | ^~~~ /usr/local/java/jdk-17.0.11/include/jni.h:1938:37: note: passing argument to parameter 'penv' here 1938 | jint AttachCurrentThread(void **penv, void *args) { | ^ /opt/AI/llm_obj/mlc-llm/3rdparty/tvm/jvm/native/src/main/native/org_apache_tvm_native_c_api.cc:309:31: error: cannot initialize a parameter of type 'void **' with an rvalue of type 'JNIEnv **' (aka 'JNIEnv_ **')

likely it depends on the JNI environment you are using, previosly android jni was directly triggred as https://github.com/apache/tvm/blob/main/jvm/native/src/main/native/org_apache_tvm_native_c_api.cc#L232 which pass in the address and normal java should not have TVM4J_ANDROID...

/opt/AI/llm_obj/mlc-llm/3rdparty/tvm/jvm/native/src/main/native/org_apache_tvm_native_c_api.cc:232:31: error: cannot initialize a parameter of type 'void **' with an rvalue of type 'JNIEnv **' (aka 'JNIEnv_ **') 232 | _jvm->AttachCurrentThread(&env, nullptr); | ^~~~ /usr/local/java/jdk-17.0.11/include/jni.h:1938:37: note: passing argument to parameter 'penv' here 1938 | jint AttachCurrentThread(void **penv, void *args) { | ^ /opt/AI/llm_obj/mlc-llm/3rdparty/tvm/jvm/native/src/main/native/org_apache_tvm_native_c_api.cc:309:31: error: cannot initialize a parameter of type 'void **' with an rvalue of type 'JNIEnv **' (aka 'JNIEnv_ **')

This is due to your JAVA_HOME setting, we highly recommend pointing to Android studio's jdk, see also https://github.com/mlc-ai/mlc-llm/pull/2327

/opt/AI/llm_obj/mlc-llm/3rdparty/tvm/jvm/native/src/main/native/org_apache_tvm_native_c_api.cc:232:31: error: cannot initialize a parameter of type 'void **' with an rvalue of type 'JNIEnv **' (aka 'JNIEnv_ **') 232 | _jvm->AttachCurrentThread(&env, nullptr); | ^~~~ /usr/local/java/jdk-17.0.11/include/jni.h:1938:37: note: passing argument to parameter 'penv' here 1938 | jint AttachCurrentThread(void **penv, void *args) { | ^ /opt/AI/llm_obj/mlc-llm/3rdparty/tvm/jvm/native/src/main/native/org_apache_tvm_native_c_api.cc:309:31: error: cannot initialize a parameter of type 'void **' with an rvalue of type 'JNIEnv **' (aka 'JNIEnv_ **')

checkout the latest instructions here https://llm.mlc.ai/docs/deploy/android.html

[Bug] `system-lib-prefix` would be cleared if `device` is not strictly `android` while `mlc_llm compile`

passing in android would be fine, and it will invoke adreno GPUs.

[Bug] `system-lib-prefix` would be cleared if `device` is not strictly `android` while `mlc_llm compile`

After reading through the code, i think @BitCircuit you are right and we can safely remove this line, do you mind send a PR?

[Bug] Error: could not compile `regex-syntax`

Seems this have to do with rust installation, consider reinstall rust environment

[Bug] LlaMa-3 doesn't work

Likely this was due to older variant of the GPU, you can try to build tvm and mlc from source without flashinfer/thrust https://llm.mlc.ai/docs/install/tvm.html

[Bug] LlaMa-3 doesn't work

we have now updated the latest to disable flashinfer if it is not available

[Bug] relax.vm.AttentionKVCache expects 19 arguments, but 18 were provided.

When you see error like this, likely it is due to stale prebuild binary. Please remove the prebuild binary - android, ios: use the latest instruction in `mlc_llm package` -...