Awni Hannun
Awni Hannun
What you have is great! Warmup is exactly right. The `.item()` is an implicit eval so you are good there. The only caveat is the timings will include the time...
Could you share your OS?
Hmm, the same test failed for me also. Curious that the CI didn't catch it.. I don't see the scan failure. Is that one consistently failing for you?
Yes the second one is more concerning for me. I can't reproduce it, but I'm hoping its a consistent failure for you o/w there may be some nondeterminism which always...
Interesting. Is it the same test that's failing for you? The slice test? Namely: ``` Traceback (most recent call last): File ".../mlx/python/tests/test_ops.py", line 1307, in test_scans self.assertTrue(mx.array_equal(c1[:, :, :-1], c2[:,...
CC @angeloskath looks like we might have some non-deterministic failure in the scan @kechan do you see this when your run the tests standalone or just running the full python...
FWIW I am able to get the test on line 1302 to fail somewhat regularly. I inspected the differences and it looks like it's a numerical tolerance thing, usually fails...
It never fails if I change the tolerances to 1e-3..
> Since it passed 100% when I don't have that inference workload, it just doesn't smell right. This was not the case for me. It fails about 50% of the...
Are you saying that it gives the wrong result when `num_heads` is not a power of `2`? Do you want to send a PR?