iOS app store validation issues

[sinkingsugar]

FYI with current master I am getting:

ITMS-90755: Invalid Binary - The following binaries in your app contains prohibited instructions: (my binary with candle in it). Remove the instructions from the binaries, rebuild and resubmit.

We are reverting to 0.8.1 for now, will update if I find out more.

[lucasjinreal]

May I ask what's the model uses here? Does it faster than llama.cpp on CPU?

[sinkingsugar]

May I ask what's the model uses here? Does it faster than llama.cpp on CPU?

Sure, we mainly use the library for tensor operations (https://github.com/fragcolor-xyz/shards/tree/devel/shards/modules/candle), as we indeed switched to use llama.cpp on GPU for most of the workloads we have. The issue is mainly for consistence and to signal lack of love for those platforms.

[lucasjinreal]

Oh, these way is strange. Does candle slower too much than llama.cpp on GPU?

[sinkingsugar]

Oh, these way is strange. Does candle slower too much than llama.cpp on GPU?

on iOS candle gpu does not even work.. currently there are some invalid metal calls that fail on iOS.

[lucasjinreal]

In that case candle team need to take a look

[sinkingsugar]

Downgrading ug to 0.0.2 fixed the issue. I suspect it's gemm that ug @ 0.1.0 uses at 0.18

[ivarflakstad]

@sinkingsugar should be fixed now :)