libc++abi: terminating due to uncaught exception of type std::runtime_error: [Event::stream] Cannot access stream on invalid event.
Running a small model on a M4 Mac and after around 60-100 epochs I keep getting: libc++abi: terminating due to uncaught exception of type std::runtime_error: [Event::stream] Cannot access stream on invalid event. The model runs completely fine at the beginning and the it keeps crashing.
Can you share code that reproduces the error you're seeing?
it's not exactly the same code but it should give you an indication of more or less what could cause this incident, I've identified a part of my script with these simple calculations and after taking them out the error no longer occurs (not sure which exact line was responsible for it):
import mlx.core as mx
arr_a[start_sequence:end_idx] += mx.where((arr_b > 50) & (arr_c > 0) & (arr_d < 500) & (var_a > 3), 1, 0) arr_a_flat = arr_a[start_idx:end_idx] arr_x = (arr_b > 50) & (arr_d < 600) arr_y = (arr_c < 20) | (arr_d >= 600) arr_z = mx.sum(arr_a_flat)
Unfortunately that snippet isn’t sufficient for us to reproduce and debug this. If you can provide a standalone program that we can run that reproduces the issue that would be great.
Please also provide some more details on your environment like OS, MLX version and hardware
Closing as inactive.
to me this happened when i had a mx.array reference leak, e.g.:
arr = mx.array([ ... big array ... ])
reg = []
for some_big_loop:
reg.append(arr[i]) # forgot to do `arr[i].item()` !
at some point this will blow up with that error.