luminal icon indicating copy to clipboard operation
luminal copied to clipboard

Phi model does not produce output on M3

Open jorgeantonio21 opened this issue 9 months ago • 4 comments

Currently, I can't extract an output by running the phi3 example:

 % cargo run --release --features metal

\    Finished release [optimized] target(s) in 0.27s
     Running `/Users/jorgeantonio/dev/luminal/target/release/phi`
Defining graph           - 75ms
Compiling graph          - 4799ms
Loading model            - 3544ms
Processing Prompt        - 183ms (71.04 tok/s, 13 prompt tokens)
<|user|>
Please write me a python implementation of merge sort<|end|>
<|assistant|>


Average token generated in 46.66ms       - (21.43 tok/s)

jorgeantonio21 avatar May 04 '24 08:05 jorgeantonio21