T-CNN
T-CNN copied to clipboard
myfavouritekk
matlab.engine.EngineError: MATLAB function cannot be evaluated
[2016-07-26 20:49:44,869 24261 greedy_tracking_from_raw_dets.py:58 INFO] Tracking lion...
[2016-07-26 20:49:45,243 24261 track.py:219 INFO] Upon low confidence: total 0 tracks
[2016-07-26 20:49:45,244 24261 greedy_tracking_from_raw_dets.py:63 WARNING] ILSVRC2015_val_00007011 has no tracks for class lion
[2016-07-26 20:49:45,244 24261 greedy_tracking_from_raw_dets.py:58 INFO] Tracking motorcycle...
[2016-07-26 20:49:45,617 24261 track.py:219 INFO] Upon low confidence: total 0 tracks
[2016-07-26 20:49:45,618 24261 greedy_tracking_from_raw_dets.py:63 WARNING] ILSVRC2015_val_00007011 has no tracks for class motorcycle
[2016-07-26 20:49:45,618 24261 greedy_tracking_from_raw_dets.py:58 INFO] Tracking cattle...
[2016-07-26 20:49:45,990 24261 track.py:219 INFO] Upon low confidence: total 0 tracks
[2016-07-26 20:49:45,991 24261 greedy_tracking_from_raw_dets.py:63 WARNING] ILSVRC2015_val_00007011 has no tracks for class cattle
[2016-07-26 20:49:45,991 24261 greedy_tracking_from_raw_dets.py:58 INFO] Tracking airplane...
[2016-07-26 20:49:46,336 24261 track.py:222 INFO] tracking top No.0 in ILSVRC2015_val_00007011
[2016-07-26 20:49:46,336 24261 common.py:341 DEBUG] Use opened Matlab session: <matlab.engine.matlabengine.MatlabEngine object at 0x7f1e308fbcd0>
[2016-07-26 20:50:08,061 24261 common.py:341 DEBUG] Use opened Matlab session: <matlab.engine.matlabengine.MatlabEngine object at 0x7f1e308fbcd0>
[2016-07-26 20:50:09,544 24261 common.py:348 ERROR] Existing engine no response: <matlab.engine.matlabengine.MatlabEngine object at 0x7f1e308fbcd0>
[2016-07-26 20:50:13,866 24261 common.py:341 DEBUG] Use opened Matlab session: <matlab.engine.matlabengine.MatlabEngine object at 0x7f1e711e4710>
[2016-07-26 20:50:29,874 24261 common.py:348 ERROR] Existing engine no response: <matlab.engine.matlabengine.MatlabEngine object at 0x7f1e711e4710>
Traceback (most recent call last):
File "tools/tracking/greedy_tracking_from_raw_dets.py", line 61, in
doses anyone happen to this? please let me know your solution, Thanks!
try adding caffe.reset_all(); to fcn_tracker.m just before caffe.set_mode_gpu();. at least for me the memory keep running out because each call of the script recreate the entire network and the matlab engine keep crashing, that's why you see this error.
@zawlin Right. reset_all() will clear stand-alone nets in the GPU. I think adding reset_all() for each fcn_tracker call may solve this problem, but it requires loading VGG for each tracking process and may takes a lot of time compared to tracking itself. Probably should call rest_all only when restarting a matlab engine. What do you think?
I got the same error. I try to add caffe.reset_all; in fcn_tracker.m, but the same crash happened. I try to add it in track.py by matlab_energy(), but I am failed. Now I don‘t know how to solve it.
[2016-10-12 11:26:06,145 10341 greedy_tracking_from_raw_dets.py:42 INFO] Video: ILSVRC2015_val_00007011
[2016-10-12 11:26:09,939 10341 greedy_tracking_from_raw_dets.py:58 INFO] Tracking turtle...
[2016-10-12 11:26:10,188 10341 track.py:219 INFO] Upon low confidence: total 0 tracks
[2016-10-12 11:26:10,190 10341 greedy_tracking_from_raw_dets.py:63 WARNING] ILSVRC2015_val_00007011 has no tracks for class turtle
[2016-10-12 11:26:10,190 10341 greedy_tracking_from_raw_dets.py:58 INFO] Tracking antelope...
[2016-10-12 11:26:10,421 10341 track.py:219 INFO] Upon low confidence: total 0 tracks
[2016-10-12 11:26:10,422 10341 greedy_tracking_from_raw_dets.py:63 WARNING] ILSVRC2015_val_00007011 has no tracks for class antelope
[2016-10-12 11:26:10,422 10341 greedy_tracking_from_raw_dets.py:58 INFO] Tracking bicycle...
[2016-10-12 11:26:10,659 10341 track.py:219 INFO] Upon low confidence: total 0 tracks
[2016-10-12 11:26:10,660 10341 greedy_tracking_from_raw_dets.py:63 WARNING] ILSVRC2015_val_00007011 has no tracks for class bicycle
[2016-10-12 11:26:10,660 10341 greedy_tracking_from_raw_dets.py:58 INFO] Tracking lion...
[2016-10-12 11:26:10,894 10341 track.py:219 INFO] Upon low confidence: total 0 tracks
[2016-10-12 11:26:10,895 10341 greedy_tracking_from_raw_dets.py:63 WARNING] ILSVRC2015_val_00007011 has no tracks for class lion
[2016-10-12 11:26:10,895 10341 greedy_tracking_from_raw_dets.py:58 INFO] Tracking motorcycle...
[2016-10-12 11:26:11,127 10341 track.py:219 INFO] Upon low confidence: total 0 tracks
[2016-10-12 11:26:11,129 10341 greedy_tracking_from_raw_dets.py:63 WARNING] ILSVRC2015_val_00007011 has no tracks for class motorcycle
[2016-10-12 11:26:11,129 10341 greedy_tracking_from_raw_dets.py:58 INFO] Tracking cattle...
[2016-10-12 11:26:11,363 10341 track.py:219 INFO] Upon low confidence: total 0 tracks
[2016-10-12 11:26:11,365 10341 greedy_tracking_from_raw_dets.py:63 WARNING] ILSVRC2015_val_00007011 has no tracks for class cattle
[2016-10-12 11:26:11,365 10341 greedy_tracking_from_raw_dets.py:58 INFO] Tracking airplane...
[2016-10-12 11:26:11,560 10341 track.py:222 INFO] tracking top No.0 in ILSVRC2015_val_00007011
[2016-10-12 11:26:11,561 10341 common.py:341 DEBUG] Use opened Matlab session: <matlab.engine.matlabengine.MatlabEngine object at 0x7f5fe9562d10>
[2016-10-12 11:26:14,244 10341 common.py:348 ERROR] Existing engine no response: <matlab.engine.matlabengine.MatlabEngine object at 0x7f5fe9562d10>
[2016-10-12 11:26:17,436 10341 common.py:341 DEBUG] Use opened Matlab session: <matlab.engine.matlabengine.MatlabEngine object at 0x7f5fe95a8550>
[2016-10-12 11:26:20,180 10341 common.py:348 ERROR] Existing engine no response: <matlab.engine.matlabengine.MatlabEngine object at 0x7f5fe95a8550>
Traceback (most recent call last):
File "tools/tracking/greedy_tracking_from_raw_dets.py", line 61, in <module>
cls_idx, opts)
File "./vdetlib/vdet/track.py", line 234, in greedily_track_from_raw_dets
new_tracks = track_method(vid_proto, anchor_frame_id, anchor_bbox, opts)
File "./vdetlib/vdet/track.py", line 82, in fcn_tracker
[matlab.double(bbox),] + fw_frames + [opts.gpu,], opts.engine)
File "./vdetlib/utils/common.py", line 349, in matlab_engine
raise e
matlab.engine.EngineError: MATLAB function cannot be evaluated
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
func(*targs, **kargs)
File "/usr/local/lib/python2.7/dist-packages/matlab/engine/__init__.py", line 186, in __exit_engines
eng().exit()
File "/usr/local/lib/python2.7/dist-packages/matlab/engine/matlabengine.py", line 246, in exit
pythonengine.closeMATLAB(self.__dict__["_matlab"])
SystemError: MATLAB process cannot be terminated.
Error in sys.exitfunc:
Traceback (most recent call last):
File "/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
func(*targs, **kargs)
File "/usr/local/lib/python2.7/dist-packages/matlab/engine/__init__.py", line 186, in __exit_engines
eng().exit()
File "/usr/local/lib/python2.7/dist-packages/matlab/engine/matlabengine.py", line 246, in exit
pythonengine.closeMATLAB(self.__dict__["_matlab"])
SystemError: MATLAB process cannot be terminated.
Exception SystemError: 'MATLAB process cannot be terminated.' in <bound method MatlabEngine.__del__ of <matlab.engine.matlabengine.MatlabEngine object at 0x7f5fe95a8550>> ignored
My GPU is GTX960M with 2GB, is it the reason? Does it mean that I should use an another caffemodel ？
I'm sorry, I didn't express it clearly @myfavouritekk . I actually try to add reset_all(); before matlab.engine.start.
For above problem, I changed my GPU into GTX1080, then added caffe.reset_all(); before caffe.set_mode_gpu(); in fcn_tracker.m. The crash didn't happen again.
I've tried adding caffe.reset_all(); before caffe.set_mode_gpu(); in fcn_tracker.m right as what @zawlin @myfavouritekk and @zong209 suggested. And my GPU is TITAN X with 12GB. But nothing changed, why?
Cleared 0 solvers and 0 stand-alone nets
[2016-12-29 19:08:59,996 8369 greedy_tracking_from_raw_dets.py:42 INFO] Video: ILSVRC2015_val_00007011
[2016-12-29 19:09:03,220 8369 greedy_tracking_from_raw_dets.py:58 INFO] Tracking turtle...
[2016-12-29 19:09:03,597 8369 track.py:219 INFO] Upon low confidence: total 0 tracks
[2016-12-29 19:09:03,599 8369 greedy_tracking_from_raw_dets.py:63 WARNING] ILSVRC2015_val_00007011 has no tracks for class turtle
[2016-12-29 19:09:03,599 8369 greedy_tracking_from_raw_dets.py:58 INFO] Tracking antelope...
[2016-12-29 19:09:03,978 8369 track.py:219 INFO] Upon low confidence: total 0 tracks
[2016-12-29 19:09:03,980 8369 greedy_tracking_from_raw_dets.py:63 WARNING] ILSVRC2015_val_00007011 has no tracks for class antelope
[2016-12-29 19:09:03,980 8369 greedy_tracking_from_raw_dets.py:58 INFO] Tracking bicycle...
[2016-12-29 19:09:04,359 8369 track.py:219 INFO] Upon low confidence: total 0 tracks
[2016-12-29 19:09:04,361 8369 greedy_tracking_from_raw_dets.py:63 WARNING] ILSVRC2015_val_00007011 has no tracks for class bicycle
[2016-12-29 19:09:04,361 8369 greedy_tracking_from_raw_dets.py:58 INFO] Tracking lion...
[2016-12-29 19:09:04,748 8369 track.py:219 INFO] Upon low confidence: total 0 tracks
[2016-12-29 19:09:04,750 8369 greedy_tracking_from_raw_dets.py:63 WARNING] ILSVRC2015_val_00007011 has no tracks for class lion
[2016-12-29 19:09:04,750 8369 greedy_tracking_from_raw_dets.py:58 INFO] Tracking motorcycle...
[2016-12-29 19:09:05,135 8369 track.py:219 INFO] Upon low confidence: total 0 tracks
[2016-12-29 19:09:05,136 8369 greedy_tracking_from_raw_dets.py:63 WARNING] ILSVRC2015_val_00007011 has no tracks for class motorcycle
[2016-12-29 19:09:05,136 8369 greedy_tracking_from_raw_dets.py:58 INFO] Tracking cattle...
[2016-12-29 19:09:05,520 8369 track.py:219 INFO] Upon low confidence: total 0 tracks
[2016-12-29 19:09:05,522 8369 greedy_tracking_from_raw_dets.py:63 WARNING] ILSVRC2015_val_00007011 has no tracks for class cattle
[2016-12-29 19:09:05,522 8369 greedy_tracking_from_raw_dets.py:58 INFO] Tracking airplane...
[2016-12-29 19:09:05,867 8369 track.py:222 INFO] tracking top No.0 in ILSVRC2015_val_00007011
[2016-12-29 19:09:05,868 8369 common.py:341 DEBUG] Use opened Matlab session: <matlab.engine.matlabengine.MatlabEngine object at 0x7f560e086190>
[2016-12-29 19:09:22,661 8369 common.py:341 DEBUG] Use opened Matlab session: <matlab.engine.matlabengine.MatlabEngine object at 0x7f5632c66ed0>
Traceback (most recent call last):
File "tools/tracking/greedy_tracking_from_raw_dets.py", line 61, in <module>
cls_idx, opts)
File "./vdetlib/vdet/track.py", line 234, in greedily_track_from_raw_dets
new_tracks = track_method(vid_proto, anchor_frame_id, anchor_bbox, opts)
File "./vdetlib/vdet/track.py", line 82, in fcn_tracker
[matlab.double(bbox),] + fw_frames + [opts.gpu,], opts.engine)
File "./vdetlib/utils/common.py", line 354, in matlab_engine
eng.clear(nargout=0)
File "/usr/local/lib/python2.7/dist-packages/matlab/engine/matlabengine.py", line 84, in __call__
_stderr).result()
File "/usr/local/lib/python2.7/dist-packages/matlab/engine/futureresult.py", line 106, in result
self._future,self._nargout, None, out=self._out, err=self._err)
matlab.engine.EngineError: MATLAB function cannot be evaluated
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
func(*targs, **kargs)
File "/usr/local/lib/python2.7/dist-packages/matlab/engine/__init__.py", line 158, in __exit_engines
eng().exit()
File "/usr/local/lib/python2.7/dist-packages/matlab/engine/matlabengine.py", line 258, in exit
pythonengine.closeMATLAB(self.__dict__["_matlab"])
SystemError: MATLAB process cannot be terminated.
Error in sys.exitfunc:
Traceback (most recent call last):
File "/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
func(*targs, **kargs)
File "/usr/local/lib/python2.7/dist-packages/matlab/engine/__init__.py", line 158, in __exit_engines
eng().exit()
File "/usr/local/lib/python2.7/dist-packages/matlab/engine/matlabengine.py", line 258, in exit
pythonengine.closeMATLAB(self.__dict__["_matlab"])
SystemError: MATLAB process cannot be terminated.
Exception SystemError: 'MATLAB process cannot be terminated.' in <bound method MatlabEngine.__del__ of <matlab.engine.matlabengine.MatlabEngine object at 0x7f5632c66ed0>> ignored
I added caffe.reset_all(); before caffe.set_mode_gpu(); in fcn_tracker.m, the problem had not solve, Can you help me?@myfavouritekk
@mingde510 or anyone else, were you able to solve this issue finally ?
I am using a GTX 1060 6 GB Variant and I get the same issue as this thread, MATLAB process cannot be terminated.
I have tried adding caffe.reset_all(); in fcn_tracker.m just before caffe.set_mode_gpu(); like @myfavouritekk, @zong209 has suggested, but my terminal output says Cleared 0 solvers and 0 stand-alone nets
And then I get the same error.
Anyone has any success?? Thanks.
