super-gradients icon indicating copy to clipboard operation
super-gradients copied to clipboard

Kernel Dead Issue

Open sivaji123256 opened this issue 2 years ago • 4 comments

Hi @BloodAxe @ofrimasad @jonathan-sha @spsancti @yurkovak , Thanks for the cool work. I was trying with the quick notebook on 5 min video.But the kernel was dead issue was rising.I couldn't run the inference. Any idea on why is happening ?

sivaji123256 avatar May 03 '23 18:05 sivaji123256

My guess that it is OOM error. In theory if the video has high resolution frames that we accumulate during processing may sum up and take all system memory. Do you mind sharing a video (or at least it's dimensions) you're trying to process?

BloodAxe avatar May 03 '23 20:05 BloodAxe

i get the same problem on yolo_nas_l.to(device).predict(input_video_path).save(output_video_path) memory get all 125gb i have and crash + 2 gb of swap

i get this working ok on the same video on yolo 7 and 8 and yolox

i am infer on yolo_nas_l

Lifeguard-alex avatar May 06 '23 17:05 Lifeguard-alex

Hi @sivaji123256 and @Lifeguard-alex , Please check out https://github.com/Deci-AI/super-gradients/issues/976, does this solve your issue ?

Louis-Dupont avatar May 11 '23 19:05 Louis-Dupont

We have implemented memory-efficient predict for video in super-gradients 3.5.0. So you are welcome to upgrade to a new version of SG which should not have this issue.

BloodAxe avatar Nov 23 '23 15:11 BloodAxe