tinygrad
tinygrad copied to clipboard
Invalid Metal library on non-conda Mac env after cmp_tuple
Mac OS, Metal, M3 Macbook Pro, Python 3.12.4 in a venv
.
I think 3929a9dc945e9c00684aa676241908550cf7b697 broke something on Metal, I am hitting an invalid library error on a non-conda environment. This was working a while ago, git bisect
points to 3929a9dc945e9c00684aa676241908550cf7b697.
AssertionError: Invalid Metal library. Could be due to using conda. Try system python or METAL_XCODE=1 DISABLE_COMPILER_CACHE=1.
To reproduce (similar to https://github.com/tinygrad/tinygrad/issues/5241):
#!/usr/bin/env python3
from tinygrad import Tensor, dtypes
import numpy as np
def main():
huge = Tensor(np.zeros((20000, 20000)), dtype=dtypes.int32)
a = Tensor.arange(20000)
b = Tensor.arange(20000)
indexed = huge[a,b]
print(indexed[0].sum().item())
if __name__ == "__main__":
main()
This works on a9d6a6c339871d489d0db63746490142e6541d82 but fails with the library error on the next commit (3929a9dc945e9c00684aa676241908550cf7b697)
(.venv) ➜ tinygrad git:(a9d6a6c3) ✗ python t.py
Traceback (most recent call last):
File "/Users/rvd/src/rvd/tinygrad/t.py", line 13, in <module>
main()
File "/Users/rvd/src/rvd/tinygrad/t.py", line 10, in main
print(indexed[0].sum().item())
^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/rvd/src/rvd/tinygrad/tinygrad/tensor.py", line 271, in item
return self._data().cast(self.dtype.fmt)[0]
^^^^^^^^^^^^
File "/Users/rvd/src/rvd/tinygrad/tinygrad/tensor.py", line 242, in _data
cpu = self.cast(self.dtype.scalar()).contiguous().to("CLANG").realize()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/rvd/src/rvd/tinygrad/tinygrad/tensor.py", line 201, in realize
run_schedule(*self.schedule_with_vars(*lst), do_update_stats=do_update_stats)
File "/Users/rvd/src/rvd/tinygrad/tinygrad/engine/realize.py", line 190, in run_schedule
for ei in lower_schedule(schedule):
File "/Users/rvd/src/rvd/tinygrad/tinygrad/engine/realize.py", line 183, in lower_schedule
while len(schedule): yield lower_schedule_item(schedule.pop(0))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/rvd/src/rvd/tinygrad/tinygrad/engine/realize.py", line 169, in lower_schedule_item
runner = get_runner(si.outputs[0].device, si.ast)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/rvd/src/rvd/tinygrad/tinygrad/engine/realize.py", line 142, in get_runner
method_cache[ckey] = method_cache[bkey] = ret = CompiledRunner(replace(prg, dname=dname))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/rvd/src/rvd/tinygrad/tinygrad/engine/realize.py", line 61, in __init__
self.clprg = Device[p.dname].runtime(p.function_name, self.lib)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/rvd/src/rvd/tinygrad/tinygrad/runtime/ops_metal.py", line 39, in __init__
assert lib[:4] == b"MTLB", "Invalid Metal library. Could be due to using conda. Try system python or METAL_XCODE=1 DISABLE_COMPILER_CACHE=1."
^^^^^^^^^^^^^^^^^^
AssertionError: Invalid Metal library. Could be due to using conda. Try system python or METAL_XCODE=1 DISABLE_COMPILER_CACHE=1.
``