Junru Shao comments

Results 179 comments of


                                            Junru Shao

[Bug] Inference for "vicuna-13b-1.1-q3f16_0" fails with "Some problems on GPU happaned!" on M2 Max 32GB

This is weird. I cannot reproduce the issue on my M2 Max. Does it work with the prebuilt Vicuna-7b?

for architecture arm64

Sorry I couldn't follow your question. Yes, we do cross-compilation to ARM64 for iOS build, because iOS is certainly ARM64.

for architecture arm64

@NullCodex on the contrary, it works for a real iphone but doesn’t work with a simulator.

[Feature Request] Decouple the Android ChatApp into library and app module

This is a great proposal and is the direction we are working towards. I'm not particularly familiar with Android, so am CC'ing some related devs @spectrometerHBH @cyx-6 @tqchen

[Bug] Unsupported opcode, application dumps core

You might want to install a more proper Vulkan driver: https://mlc.ai/mlc-llm/docs/install/software-dependencies.html#valkan-driver-validate-installation

[Bug] Unsupported opcode, application dumps core

@GTP95 how about this one: https://mlc.ai/mlc-llm/docs/install/gpu.html

ARM64 CPU on Linux

Seems that you are not using the correct TVM branch. Would you mind double checking: https://mlc.ai/mlc-llm/docs/install/tvm.html#id2

[Question] TVM version used in tvm wheels?

Hi Ziyu, long time no see! You may find our documentation helpful here: https://mlc.ai/mlc-llm/docs/install/tvm.html#id2

[Question] TVM version used in tvm wheels?

Closing as there is nothing actionable further

It Just Seemingly Crashes....?

4G VRAM is not enough for Vicuna-7b.