luminal
luminal copied to clipboard
Phi model does not produce output on M3
Currently, I can't extract an output by running the phi3 example:
% cargo run --release --features metal
\ Finished release [optimized] target(s) in 0.27s
Running `/Users/jorgeantonio/dev/luminal/target/release/phi`
Defining graph - 75ms
Compiling graph - 4799ms
Loading model - 3544ms
Processing Prompt - 183ms (71.04 tok/s, 13 prompt tokens)
<|user|>
Please write me a python implementation of merge sort<|end|>
<|assistant|>
Average token generated in 46.66ms - (21.43 tok/s)