MDP_Tracking icon indicating copy to clipboard operation
MDP_Tracking copied to clipboard

Published 20 hours ago • verified publisher icon yuxng

Out of Memory running cross validation

Open omrishiv opened this issue 8 years ago • 13 comments
trafficstars

Hello, I downloaded your code and am trying to use it. I was having issues with the imResize function and increased my maximum array size; however, I am now running into out of memory issues; how much memory should be required to run this?

omrishiv avatar Apr 06 '17 17:04 omrishiv

Were you able to solve it ? I am having similar issue, though it is due to imResampleMex Error using imResampleMex Requested 863288426630x139672336465921x139675000218880 (17179869184.0GB) array exceeds maximum array size preference. Creation of arrays greater than this limit may take a long time and cause MATLAB to become unresponsive. See array size limit or preference panel for more information

as6520 avatar Jun 12 '17 19:06 as6520

Same as as6520 above, imResampleMex error trying to run (specifically train) MOT_test.m

Error using imResampleMex Requested 863288426630x139702401236993x139706387807888 (17179869184.0GB) array exceeds maximum array size preference. Creation of arrays greater than this limit may take a long time and cause MATLAB to become unresponsive. See array size limit or preference panel for more information.

Worth noting that 17179869184 = 2^34, and that's much more memory than I have.

ConlonNovak avatar Jun 12 '17 19:06 ConlonNovak

I have the same issue with MOT-test and MOT_cross_validation Has anyone solved this issue?

CamilleTarsiguel avatar Jun 29 '17 09:06 CamilleTarsiguel

Not a fix but I was able to run both test and validation after changing method used by imResample from 'bilinear' to 'nearest' in LK_crop_image_box.m

as6520 avatar Jun 29 '17 13:06 as6520

Indeed thank you!

CamilleTarsiguel avatar Jun 30 '17 11:06 CamilleTarsiguel

@as6520 did you get the same results as the paper? By changing "bilinear" to "nearest", I couldn't get the same results. I tried to redo the experiment of Fig.5. But there is too much difference between my results and theirs. For example, by training on PETS09-S2L1 and testing on ADL-Rundle-8 I will get -15.1, while their reported result is 22.1. (MOTA)

namt324 avatar Jul 04 '17 19:07 namt324

Any solution to this problem yet? I'm having the same issue. I need to replicate the results of the paper, so replacing "bilinear" to "nearest" is not an option for me.

oscfri avatar Oct 10 '17 01:10 oscfri

I think I found the source of the error. ns, ms and nch in mexFunction of imResampleMex.cpp should have type size_t instead of int. Guess it is because of 32 bit/ 64 bit inconsistencies.

oscfri avatar Oct 11 '17 01:10 oscfri

@namt324 I ran MOT_cross_validation.m (using the default 2D_MOT_2015 dataset) after changing "bilinear" to "nearest" in LK_crop_image_box.m too. I got a decent MOTA of 19.8 .

	... ADL-Rundle-8
*** 2D (Bounding Box overlap) ***
 Rcll  Prcn   FAR|  GT   MT   PT   ML|    FP     FN   IDs    FM|  MOTA  MOTP 
 46.0  64.1  2.67|  28    6   13    9|  1749   3665    26   101|  19.8  72.5

BadWindshield avatar Nov 23 '17 04:11 BadWindshield

I got the same problem before and I find the solution now. it is because of the 32 bit/ 64 bit inconsistencies. So you just need to do a little bit change in compile.m as follow:

... eval(['mex -DMX_COMPAT_32 lk.cpp -O' include lib]); mex -DMX_COMPAT_32 distance.cpp mex -DMX_COMPAT_32 imResampleMex.cpp mex -DMX_COMPAT_32 warp.cpp ...

And everything works for me now :D

XinyingWang55 avatar Dec 05 '17 05:12 XinyingWang55

Hello,

I had the same problem,

>> MOT_test
Training on sequence: TUD-Stadtmitte
load images from file results/TUD-Stadtmitte_dres_image.mat done
TUD-Stadtmitte: 10 positive sequences
initialize tracker from scratch
.Error using imResampleMex
Requested 863288426630x1x34 (27336.0GB) array exceeds maximum array size preference. Creation of arrays greater than this limit may take a long
time and cause MATLAB to become unresponsive. See array size limit or preference panel for more information.

Error in imResample (line 51)
  B=imResampleMex(A,m1,n1,norm);

Error in LK_crop_image_box (line 16)
I_scale = imResample(I, imsize, 'bilinear');

Error in LK_initialize (line 43)
    [I_crop, BB_crop] = LK_crop_image_box(I, BB, tracker);

Error in MDP_train (line 172)
            tracker = LK_initialize(tracker, fr, id, dres, ind, dres_image);

Error in MOT_test (line 39)
            tracker = MDP_train(idx_train{j}, tracker);

I tried XinyingWang55's comment and I'm getting different error.

I use MATLAB R2017b, gcc 5.4.0 opencv 3.4.0

>> MOT_test
Training on sequence: TUD-Stadtmitte
load images from file results/TUD-Stadtmitte_dres_image.mat done
TUD-Stadtmitte: 10 positive sequences
initialize tracker from scratch
.Invalid MEX-file '/home/ahyun/cvlab/MDP_Tracking/lk.mexa64':
Missing symbol '_ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE4findEcm' required by
'/usr/local/lib/libopencv_core.so.3.4->/home/ahyun/cvlab/MDP_Tracking/lk.mexa64'
Missing symbol '_ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE4findEPKcmm' required by
'/usr/local/lib/libopencv_core.so.3.4->/home/ahyun/cvlab/MDP_Tracking/lk.mexa64'
Missing symbol '_ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE5rfindEcm' required by
'/usr/local/lib/libopencv_core.so.3.4->/home/ahyun/cvlab/MDP_Tracking/lk.mexa64'
Missing symbol '_ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE7compareEPKc' required by
'/usr/local/lib/libopencv_core.so.3.4->/home/ahyun/cvlab/MDP_Tracking/lk.mexa64'
Missing symbol '_ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE7compareERKS4_' required by
'/usr/local/lib/libopencv_core.so.3.4->/home/ahyun/cvlab/MDP_Tracking/lk.mexa64'
Missing symbol '_ZNKSt7__cxx1115basic_stringbufIcSt11char_traitsIcESaIcEE3strEv' required by
'/usr/local/lib/libopencv_core.so.3.4->/home/ahyun/cvlab/MDP_Tracking/lk.mexa64'
Missing symbol '_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE10_M_replaceEmmPKcm' required by
'/usr/local/lib/libopencv_core.so.3.4->/home/ahyun/cvlab/MDP_Tracking/lk.mexa64'
Missing symbol '_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE14_M_replace_auxEmmmc' required by
'/usr/local/lib/libopencv_core.so.3.4->/home/ahyun/cvlab/MDP_Tracking/lk.mexa64'
Missing symbol '_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE7reserveEm' required by
'/usr/local/lib/libopencv_core.so.3.4->/home/ahyun/cvlab/MDP_Tracking/lk.mexa64'
Missing symbol '_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_appendEPKcm' required by
'/usr/local/lib/libopencv_core.so.3.4->/home/ahyun/cvlab/MDP_Tracking/lk.mexa64'
Missing symbol '_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_assignERKS4_' required by
'/usr/local/lib/libopencv_core.so.3.4->/home/ahyun/cvlab/MDP_Tracking/lk.mexa64'
Missing symbol '_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_createERmm' required by
'/usr/local/lib/libopencv_core.so.3.4->/home/ahyun/cvlab/MDP_Tracking/lk.mexa64'
Missing symbol '_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_mutateEmmPKcm' required by
'/usr/local/lib/libopencv_core.so.3.4->/home/ahyun/cvlab/MDP_Tracking/lk.mexa64'
Missing symbol '_ZNSt7__cxx1115basic_stringbufIcSt11char_traitsIcESaIcEE7_M_syncEPcmm' required by
'/usr/local/lib/libopencv_core.so.3.4->/home/ahyun/cvlab/MDP_Tracking/lk.mexa64'
Missing symbol '_ZNSt7__cxx1118basic_stringstreamIcSt11char_traitsIcESaIcEEC1ESt13_Ios_Openmode' required by
'/usr/local/lib/libopencv_core.so.3.4->/home/ahyun/cvlab/MDP_Tracking/lk.mexa64'
Missing symbol '_ZNSt7__cxx1118basic_stringstreamIcSt11char_traitsIcESaIcEED1Ev' required by
'/usr/local/lib/libopencv_core.so.3.4->/home/ahyun/cvlab/MDP_Tracking/lk.mexa64'
Missing symbol '_ZNSt7__cxx1119basic_istringstreamIcSt11char_traitsIcESaIcEED1Ev' required by
'/usr/local/lib/libopencv_core.so.3.4->/home/ahyun/cvlab/MDP_Tracking/lk.mexa64'
Missing symbol '_ZNSt7__cxx1119basic_ostringstreamIcSt11char_traitsIcESaIcEEC1ESt13_Ios_Openmode' required by
'/usr/local/lib/libopencv_core.so.3.4->/home/ahyun/cvlab/MDP_Tracking/lk.mexa64'
Missing symbol '_ZNSt7__cxx1119basic_ostringstreamIcSt11char_traitsIcESaIcEED1Ev' required by
'/usr/local/lib/libopencv_core.so.3.4->/home/ahyun/cvlab/MDP_Tracking/lk.mexa64'
Missing symbol '_ZSt7getlineIcSt11char_traitsIcESaIcEERSt13basic_istreamIT_T0_ES7_RNSt7__cxx1112basic_stringIS4_S5_T1_EES4_' required by
'/usr/local/lib/libopencv_core.so.3.4->/home/ahyun/cvlab/MDP_Tracking/lk.mexa64'
Missing symbol '_ZStrsIcSt11char_traitsIcESaIcEERSt13basic_istreamIT_T0_ES7_RNSt7__cxx1112basic_stringIS4_S5_T1_EE' required by
'/usr/local/lib/libopencv_core.so.3.4->/home/ahyun/cvlab/MDP_Tracking/lk.mexa64'
Missing symbol '_ZTTNSt7__cxx1118basic_stringstreamIcSt11char_traitsIcESaIcEEE' required by
'/usr/local/lib/libopencv_core.so.3.4->/home/ahyun/cvlab/MDP_Tracking/lk.mexa64'
Missing symbol '_ZTTNSt7__cxx1119basic_istringstreamIcSt11char_traitsIcESaIcEEE' required by
'/usr/local/lib/libopencv_core.so.3.4->/home/ahyun/cvlab/MDP_Tracking/lk.mexa64'
Missing symbol '_ZTTNSt7__cxx1119basic_ostringstreamIcSt11char_traitsIcESaIcEEE' required by
'/usr/local/lib/libopencv_core.so.3.4->/home/ahyun/cvlab/MDP_Tracking/lk.mexa64'
Missing symbol '_ZTVNSt7__cxx1115basic_stringbufIcSt11char_traitsIcESaIcEEE' required by
'/usr/local/lib/libopencv_core.so.3.4->/home/ahyun/cvlab/MDP_Tracking/lk.mexa64'
Missing symbol '_ZTVNSt7__cxx1118basic_stringstreamIcSt11char_traitsIcESaIcEEE' required by
'/usr/local/lib/libopencv_core.so.3.4->/home/ahyun/cvlab/MDP_Tracking/lk.mexa64'
Missing symbol '_ZTVNSt7__cxx1119basic_istringstreamIcSt11char_traitsIcESaIcEEE' required by
'/usr/local/lib/libopencv_core.so.3.4->/home/ahyun/cvlab/MDP_Tracking/lk.mexa64'
Missing symbol '_ZTVNSt7__cxx1119basic_ostringstreamIcSt11char_traitsIcESaIcEEE' required by
'/usr/local/lib/libopencv_core.so.3.4->/home/ahyun/cvlab/MDP_Tracking/lk.mexa64'.

Error in LK (line 23)
xFJ = lk(2, I, J, xFI, xFII, level);

Error in LK_tracking (line 37)
    [BB2, xFJ, flag, medFB, medNCC, medFB_left, medFB_right, medFB_up, medFB_down] = LK(I_crop, J_crop, ...

Error in MDP_feature_tracked (line 12)
tracker = LK_tracking(frame_id, dres_image, dres_det, tracker);

Error in MDP_value (line 14)
    [tracker, f] = MDP_feature_tracked(frame_id, dres_image, dres_det, tracker);

Error in MDP_train (line 185)
            tracker = MDP_value(tracker, fr, dres_image, dres, []);

Error in MOT_test (line 39)
            tracker = MDP_train(idx_train{j}, tracker);

Any solution for this?

+)I solved this by https://github.com/vlfeat/matconvnet/issues/967#issuecomment-298167606

ahyunSeo avatar Feb 26 '18 12:02 ahyunSeo

I met the same issue when running "MOT_cross_validation.m", the error is:

.Error using imResampleMex
Requested 863288426630x14163112742591922177x3022486034818219488 (17179869184.0GB) array exceeds maximum array size preference.
Creation of arrays greater than this limit may take a long time and cause MATLAB to become unresponsive. See array size limit or
preference panel for more information.

I am not sure if there has something to do with the version of open-cv. Right now I am using version 3.4.3

I used the way mentioned by @oscfri , then this error got fix. Looks it relates to the inconsistent of size_t and int in 64bit environment.

huahandsome avatar Sep 07 '18 19:09 huahandsome

@namt324 I ran MOT_cross_validation.m (using the default 2D_MOT_2015 dataset) after changing "bilinear" to "nearest" in LK_crop_image_box.m too. I got a decent MOTA of 19.8 .

	... ADL-Rundle-8
*** 2D (Bounding Box overlap) ***
 Rcll  Prcn   FAR|  GT   MT   PT   ML|    FP     FN   IDs    FM|  MOTA  MOTP 
 46.0  64.1  2.67|  28    6   13    9|  1749   3665    26   101|  19.8  72.5

您好， ，您这边调试成功了吗？我想请教一下您这边是怎么做的，可以吗？

yeqingQian avatar Mar 06 '19 08:03 yeqingQian