exo-explore
Not detecting M1 Mac. Possible Bug
Raising an issue as @AlexCheema suggested. M1 Mac not being detected.
Error: Failed to fetch completions: Error processing prompt (see logs with DEBUG>=2): <AioRpcError of RPC that terminated with: status = StatusCode.UNKNOWN details = "Unexpected <class \IngramDeviceCompileError>: Error Domain=MTLLibraryErrorDomain Code=3 error: unknown type name 'bfloat'; did you mean 'float'? kernel void E_131072_32_4n4(device half* data0, const device bfloat* data1, uint3 gid [[threadgroup_position_in_grid]], uint3 lid [[thread_position_in_threadgroup]]) { ^~~~~~~~ float program_source:7:3: error: use of undeclared identifier 'bfloat'; did you mean 'float'? bfloat val0 = *(data1+alu0+2); ^~~~~~~~ float program_source:9.3: error: use of undeclared identifier 'bfloat'; did you mean 'float'? bfloat val1 = *(data1+alu0+2); ^~~~~~~~ float program_source:10:3: error: use of undeclared identifier 'bfloat'; did you mean 'float'? bfloat val2 = (data1+alu0+3); ^~~~~~~~ float program_source:10:3: error: use of undeclared identifier 'bfloat'; did you mean 'float'? bfloat val3 = (data1+alu0); ^~~~~~~~ float" UserInfo=(NSLocalizedDescription=program_source:3.62: error: unknown type name 'bfloat'; did you mean 'float'? kernel void E_131072_32_4n4(device half data0, const device bfloat data1, uint3 gid [[threadgroup_position_in_grid]], uint3 lid [[thread_position_in_threadgroup]]) { ^~~~~~~~ float program_source:7.3: error: use of undeclared identifier 'bfloat'; did you mean 'float'? bfloat val0 = *(data1+alu0+2); ^~~~~~~~ float program_source:9.3: error: use of undeclared identifier 'bfloat'; did you mean 'float'? bfloat val1 = *(data1+alu0+2); ^~~~~~~~ float program_source:10.3: error: use of undeclared identifier 'bfloat'; did you mean 'float'? bfloat val2 = (data1+alu0+3); ^~~~~~~~ float program_source:10:3: error: use of undeclared identifier 'bfloat'; did you mean 'float'? bfloat val3 = (data1+alu0); ^~~~~~~~ float }) @date(1611:24.09.093256-08:00)" grpc_status:2, grpc_message:"Unexpected <class \IngramDeviceCompileError>: Error Domain=MTLLibraryErrorDomain Code=3 program_source:3:62: error: unknown type name 'bfloat'; did you mean 'float'? kernel void E_131072_32_4n4(device half data0, const device bfloat data1, uint3 gid [[threadgroup_position_in_grid]], uint3 lid [[thread_position_in_threadgroup]]) { ^~~~~~~~ float\nprogram_source:7:3: error: use of undeclared identifier 'bfloat'; did you mean 'float'?\n bfloat val0 = *(data1+alu0+1);\n ^~~~~~~~ float\nprogram_source:9:3: error: use of undeclared identifier 'bfloat'; did you mean 'float'?\n bfloat val1 = *(data1+alu0+2);\n ^~~~~~~~ float\nprogram_source:10:3: error: use of undeclared identifier 'bfloat'; did you mean 'float'?\n bfloat val2 = (data1+alu0+3);\n ^~~~~~~~ float\nprogram_source:10:3: error: use of undeclared identifier 'bfloat'; did you mean 'float'?\n bfloat val3 = (data1+alu0);\n ^~~~~~~~ float" UserInfo=(NSLocalizedDescription=program_source:3:62: error: unknown type name 'bfloat'; did you mean 'float'?kernel void E_131072_32_4n4(device half data0, const device bfloat data1, uint3 gid [[threadgroup_position_in_grid]], uint3 lid [[thread_position_in_threadgroup]]) {\n ^~~~~~~~ float\nprogram_source:7:3: error: use of undeclared identifier 'bfloat'; did you mean 'float'?\n bfloat val0 = *(data1+alu0+1);\n ^~~~~~~~ float\nprogram_source:9:3: error: use of undeclared identifier 'bfloat'; did you mean 'float'?\n bfloat val1 = *(data1+alu0+2);\n ^~~~~~~~ float\nprogram_source:9:3: error: use of undeclared identifier 'bfloat'; did you mean 'float'?\n bfloat val2 = *(data1+alu0+3);\n ^~~~~~~~ float\nprogram_source:9:3: error: use of undeclared identifier 'bfloat'; did you mean 'float'?\n bfloat val3 = *(data1+alu0);\n ^~~~~~~~ float))?> >
ok... just wanted to check. I recently ran into an issue on an M1 MacBook Air that hadn't been updated in a while and upgrading the OS fixed it, but that's obviously not the problem here.
You can force it to use MLX by running exo like this: exo --inference-engine mlx
Does that work for you?
Still need to fix detection of Apple Silicon ofc but at least we can narrow it down if this works.
Hey @AlexCheema thank you for replying to my issue. When I tried implementing your solution, I ran into some new errors which I've attached here. These errors keep showing up specifically when I run this on my Node 2 M1 Mac terminal using exo. When I run your suggested command on my node 1 M3 mac exo runs right away. On my Node 2 M1 Mac exo is having trouble recognizing my Mac as a M1 Mac. It keeps thinking its an Intel Mac when its not. Maybe thats why MLX is not being detected by exo? Please let me know what you think might be causing this. Thank you.
Can you please run this @smokk89
python -c "import sys; print('Platform:', sys.platform); import platform; print('Machine:', platform.machine())"
@AlexCheema Thank you just ran it. Really weird how I am seeing X86 as machine on my M1 Mac but M3 is showing Arm as machine. Any reason why this is happening?
@AlexCheema Thank you just ran it. Really weird how I am seeing X86 as machine on my M1 Mac but M3 is showing Arm as machine. Any reason why this is happening?
That would explain the issues you're running into...
I think you may be using Rosetta 2 (Apple's x86_64 emulation layer) to run Python.
Try installing Python again:
arch -arm64 brew install python
Check which version of python you're using with:
file $(which python3)
Closing as above suggestion should fix this. Please re-open if you still have this issue.
I can also reproduce this issue on my M2 Pro Mac mini. It is due to the brew not finding the correct 'arm64' architecture of the mac os.
✘ ~ brew config
......
macOS: 15.3.2-x86_64
.....
✘ ~ python -c "import sys; print('Platform:', sys.platform); import platform; print('Machine:', platform.machine())"
Platform: darwin
Machine: arm64
~ python --version
Python 3.9.6
~ python3.12 -c "import sys; print('Platform:', sys.platform); import platform; print('Machine:', platform.machine())"
Platform: darwin
Machine: x86_64
