Tianqi Chen
Tianqi Chen
would be great if you can check https://webgpureport.org/ and send a screen shot, it may have to do with how we order adapters
this is a good suggestion, @hykilpikonna do you mind send a PR to fix this
Likely the self speculating models like eagle would help in this case
Unfortunately the SDK relies on GPU so do not work on simulator for now. One way to run might be to build for iPad on Mac.
I am not that familar with iOS part of such dev, but the original swift source is here https://github.com/mlc-ai/mlc-llm/tree/main/ios/MLCSwift, feel free to take a look and see if any change...
To build an macapp, likely an update on https://github.com/mlc-ai/mlc-llm/blob/main/ios/prepare_libs.sh is needed. The swift UI also need some update, if you manage to do it, we love a PR
I would avoid multiarch build, mainly because the the .o file that we build have a specific target string https://github.com/mlc-ai/mlc-llm/blob/main/mlc_llm/utils.py#L435 which should work for M1(arch64) but not for x86. For...
I think the main issue would be multi arch. If we build the app for either apple silicon or x86 I think updating the flag should work. So atm we...
Aka if we set the right target and say build an app only for apple silicon MacOS. I think the main thing would be to update the mlc related build...
We depend on tvm unity (the latest developments), please checkout instructions here https://mlc.ai/mlc-llm/docs/install/tvm.html