PSPNet
PSPNet copied to clipboard
hszhao
Hi, GPU GTX 1070 8G memory is not enough?
Hi, thanks for your work, I use GTX 1070 8G memory to run the eval_all.m. But it tell me out of memory. Is it not enough?
The error is：
processing 1 (500)... F1013 10:40:02.524335 4443 syncedmem.cpp:56] Check failed: error == cudaSuccess (2 vs. 0) out of memory *** Check failure stack trace: ***
abort() detected at Sat Oct 13 10:40:02 2018
Configuration: Crash Decoding : Disabled Current Visual : None Default Encoding : UTF-8 GNU C Library : 2.23 stable MATLAB Architecture: glnxa64 MATLAB Root : /usr/local/MATLAB/R2014a MATLAB Version : 8.3.0.532 (R2014a) Operating System : Linux 4.15.0-36-generic #39~16.04.1-Ubuntu SMP Tue Sep 25 08:59:23 UTC 2018 x86_64 Processor ID : x86 Family 6 Model 158 Stepping 10, GenuineIntel Virtual Machine : Java 1.7.0_11-b21 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode Window System : No active display
Fault Count: 1
Abnormal termination: abort()
Register State (from fault): RAX = 0000000000000000 RBX = 00007fdbbc5f8420 RCX = 00007fdc42d90428 RDX = 0000000000000006 RSP = 00007fdc2bfee848 RBP = 00007fdc2bfeeb20 RSI = 000000000000115b RDI = 0000000000001119
R8 = 0000000000000081 R9 = 0000000000000000 R10 = 0000000000000008 R11 = 0000000000000202 R12 = 00007fdbbc5f8480 R13 = 000000000000006a R14 = 00007fdbbc5f8420 R15 = 00007fdbbc5ffde0
RIP = 00007fdc42d90428 EFL = 0000000000000202
CS = 0033 FS = 0000 GS = 0000
Stack Trace (from fault): [ 0] 0x00007fdc42d90428 /lib/x86_64-linux-gnu/libc.so.6+00218152 gsignal+00000056 [ 1] 0x00007fdc42d9202a /lib/x86_64-linux-gnu/libc.so.6+00225322 abort+00000362 [ 2] 0x00007fdbbc3e3e49 /usr/lib/x86_64-linux-gnu/libglog.so.0+00040521 [ 3] 0x00007fdbbc3e55cd /usr/lib/x86_64-linux-gnu/libglog.so.0+00046541 [ 4] 0x00007fdbbc3e7433 /usr/lib/x86_64-linux-gnu/libglog.so.0+00054323 ZN6google10LogMessage9SendToLogEv+00000643 [ 5] 0x00007fdbbc3e515b /usr/lib/x86_64-linux-gnu/libglog.so.0+00045403 ZN6google10LogMessage5FlushEv+00000187 [ 6] 0x00007fdbbc3e7e1e /usr/lib/x86_64-linux-gnu/libglog.so.0+00056862 ZN6google15LogMessageFatalD2Ev+00000014 [ 7] 0x00007fdbc3439380 /home/leequens/PSPNet/matlab/+caffe/private/caffe.mexa64+02823040 [ 8] 0x00007fdbc3438349 /home/leequens/PSPNet/matlab/+caffe/private/caffe.mexa64+02818889 [ 9] 0x00007fdbc3442a72 /home/leequens/PSPNet/matlab/+caffe/private/caffe.mexa64+02861682 [ 10] 0x00007fdbc33be93f /home/leequens/PSPNet/matlab/+caffe/private/caffe_.mexa64+02320703 [ 11] 0x00007fdbc324ff32 /home/leequens/PSPNet/matlab/+caffe/private/caffe_.mexa64+00818994 [ 12] 0x00007fdbc3250086 /home/leequens/PSPNet/matlab/+caffe/private/caffe_.mexa64+00819334 [ 13] 0x00007fdbc31d6fe2 /home/leequens/PSPNet/matlab/+caffe/private/caffe_.mexa64+00323554 [ 14] 0x00007fdbc31d811c /home/leequens/PSPNet/matlab/+caffe/private/caffe_.mexa64+00327964 mexFunction+00000169 [ 15] 0x00007fdc3acbd72a /usr/local/MATLAB/R2014a/bin/glnxa64/libmex.so+00120618 mexRunMexFile+00000090 [ 16] 0x00007fdc3acb9a94 /usr/local/MATLAB/R2014a/bin/glnxa64/libmex.so+00105108 [ 17] 0x00007fdc3acbafb4 /usr/local/MATLAB/R2014a/bin/glnxa64/libmex.so+00110516 [ 18] 0x00007fdc3a0b4ad9 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_dispatcher.so+00670425 ZN8Mfh_file11dispatch_fhEiPP11mxArray_tagiS2+00000697 [ 19] 0x00007fdc3918b20e /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02601486 [ 20] 0x00007fdc391461d0 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02318800 [ 21] 0x00007fdc391481ea /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02327018 [ 22] 0x00007fdc3914b167 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02339175 [ 23] 0x00007fdc3914926f /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02331247 [ 24] 0x00007fdc39149ec4 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02334404 [ 25] 0x00007fdc391a730b /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02716427 [ 26] 0x00007fdc3a0b4ad9 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_dispatcher.so+00670425 ZN8Mfh_file11dispatch_fhEiPP11mxArray_tagiS2+00000697 [ 27] 0x00007fdc39cec6e8 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwmcos.so+01566440 [ 28] 0x00007fdc39c96a02 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwmcos.so+01214978 [ 29] 0x00007fdc39c9838e /usr/local/MATLAB/R2014a/bin/glnxa64/libmwmcos.so+01221518 [ 30] 0x00007fdc39c9ae50 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwmcos.so+01232464 [ 31] 0x00007fdc39c9873d /usr/local/MATLAB/R2014a/bin/glnxa64/libmwmcos.so+01222461 [ 32] 0x00007fdc39d5a1f0 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwmcos.so+02015728 [ 33] 0x00007fdc3a063874 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_dispatcher.so+00338036 ZN13Mfh_MATLAB_fn11dispatch_fhEiPP11mxArray_tagiS2+00000244 [ 34] 0x00007fdc39d5b031 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwmcos.so+02019377 [ 35] 0x00007fdc3918ae4e /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02600526 [ 36] 0x00007fdc3919e524 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02680100 [ 37] 0x00007fdc3919e9f0 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02681328 [ 38] 0x00007fdc3919f9d6 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02685398 [ 39] 0x00007fdc39131046 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02232390 [ 40] 0x00007fdc3912ba15 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02210325 [ 41] 0x00007fdc39147233 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02322995 [ 42] 0x00007fdc3914b167 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02339175 [ 43] 0x00007fdc3914926f /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02331247 [ 44] 0x00007fdc39149ec4 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02334404 [ 45] 0x00007fdc391a730b /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02716427 [ 46] 0x00007fdc3a0b4ad9 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_dispatcher.so+00670425 ZN8Mfh_file11dispatch_fhEiPP11mxArray_tagiS2+00000697 [ 47] 0x00007fdc39cec6e8 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwmcos.so+01566440 [ 48] 0x00007fdc39c96a02 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwmcos.so+01214978 [ 49] 0x00007fdc39c9838e /usr/local/MATLAB/R2014a/bin/glnxa64/libmwmcos.so+01221518 [ 50] 0x00007fdc39c9ae50 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwmcos.so+01232464 [ 51] 0x00007fdc39c9873d /usr/local/MATLAB/R2014a/bin/glnxa64/libmwmcos.so+01222461 [ 52] 0x00007fdc39d5a1f0 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwmcos.so+02015728 [ 53] 0x00007fdc3a063874 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_dispatcher.so+00338036 ZN13Mfh_MATLAB_fn11dispatch_fhEiPP11mxArray_tagiS2+00000244 [ 54] 0x00007fdc39d5b031 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwmcos.so+02019377 [ 55] 0x00007fdc3918ae4e /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02600526 [ 56] 0x00007fdc3919e524 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02680100 [ 57] 0x00007fdc3919e9f0 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02681328 [ 58] 0x00007fdc3919f9d6 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02685398 [ 59] 0x00007fdc39131046 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02232390 [ 60] 0x00007fdc3912ba15 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02210325 [ 61] 0x00007fdc39147233 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02322995 [ 62] 0x00007fdc3914b167 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02339175 [ 63] 0x00007fdc3914926f /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02331247 [ 64] 0x00007fdc39149ec4 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02334404 [ 65] 0x00007fdc391a730b /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02716427 [ 66] 0x00007fdc3a0b4ad9 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_dispatcher.so+00670425 ZN8Mfh_file11dispatch_fhEiPP11mxArray_tagiS2+00000697 [ 67] 0x00007fdc3918b20e /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02601486 [ 68] 0x00007fdc391461d0 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02318800 [ 69] 0x00007fdc391481ea /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02327018 [ 70] 0x00007fdc3914b167 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02339175 [ 71] 0x00007fdc3914926f /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02331247 [ 72] 0x00007fdc39149ec4 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02334404 [ 73] 0x00007fdc391a730b /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02716427 [ 74] 0x00007fdc3a0b4ad9 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_dispatcher.so+00670425 ZN8Mfh_file11dispatch_fhEiPP11mxArray_tagiS2+00000697 [ 75] 0x00007fdc3918b20e /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02601486 [ 76] 0x00007fdc391461d0 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02318800 [ 77] 0x00007fdc391481ea /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02327018 [ 78] 0x00007fdc3914b167 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02339175 [ 79] 0x00007fdc3914926f /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02331247 [ 80] 0x00007fdc39149ec4 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02334404 [ 81] 0x00007fdc391a730b /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02716427 [ 82] 0x00007fdc3a0b4ad9 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_dispatcher.so+00670425 ZN8Mfh_file11dispatch_fhEiPP11mxArray_tagiS2+00000697 [ 83] 0x00007fdc393512b4 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+04461236 [ 84] 0x00007fdc39352bc9 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+04467657 [ 85] 0x00007fdc393533fc /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+04469756 [ 86] 0x00007fdc391cd6e3 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02873059 [ 87] 0x00007fdc391dd09e /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02936990 [ 88] 0x00007fdc391dd183 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02937219 [ 89] 0x00007fdc39313172 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+04206962 [ 90] 0x00007fdc39148589 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02327945 [ 91] 0x00007fdc3914b167 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02339175 [ 92] 0x00007fdc3914926f /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02331247 [ 93] 0x00007fdc39149ec4 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02334404 [ 94] 0x00007fdc391a730b /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02716427 [ 95] 0x00007fdc3a0b4c5f /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_dispatcher.so+00670815 ZN8Mfh_file11dispatch_fhEiPP11mxArray_tagiS2+00001087 [ 96] 0x00007fdc393512b4 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+04461236 [ 97] 0x00007fdc39352bc9 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+04467657 [ 98] 0x00007fdc393533fc /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+04469756 [ 99] 0x00007fdc391cd6e3 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02873059 [100] 0x00007fdc391dd09e /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02936990 [101] 0x00007fdc391dd183 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02937219 [102] 0x00007fdc39313172 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+04206962 [103] 0x00007fdc39149df8 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02334200 [104] 0x00007fdc391a730b /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02716427 [105] 0x00007fdc3a0b4c5f /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_dispatcher.so+00670815 ZN8Mfh_file11dispatch_fhEiPP11mxArray_tagiS2+00001087 [106] 0x00007fdc3917a135 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02531637 [107] 0x00007fdc391410d9 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02298073 [108] 0x00007fdc3913e9e1 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02288097 [109] 0x00007fdc3913ebf1 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02288625 [110] 0x00007fdc39140290 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+02294416 [111] 0x00007fdc390e5a75 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwm_interpreter.so+01923701 _Z51inEvalCmdWithLocalReturnInDesiredWSAndPublishEventsPKcPibbP15inWorkSpace_tag+00000085 [112] 0x00007fdc3b19ce68 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwiqm.so+00650856 _ZNK3iqm18InternalEvalPlugin24inEvalCmdWithLocalReturnERKSbIDsSt11char_traitsIDsESaIDsEEP15inWorkSpace_tag+00000120 [113] 0x00007fdc3b19d328 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwiqm.so+00652072 _ZN3iqm18InternalEvalPlugin7executeEP15inWorkSpace_tagRN5boost10shared_ptrIN14cmddistributor17IIPCompletedEventEEE+00000120 [114] 0x00007fdc442120c5 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwmcr.so+00434373 [115] 0x00007fdc3b18f565 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwiqm.so+00595301 [116] 0x00007fdc3b17db05 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwiqm.so+00523013 [117] 0x00007fdc3aee2ad9 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwbridge.so+00121561 _Z10ioReadLinebP8_IO_FILERKN5boost8optionalIKP15inWorkSpace_tagEEb+00000345 [118] 0x00007fdc3aee3081 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwbridge.so+00123009 [119] 0x00007fdc3aee7c2d /usr/local/MATLAB/R2014a/bin/glnxa64/libmwbridge.so+00142381 [120] 0x00007fdc3aee7fe9 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwbridge.so+00143337 [121] 0x00007fdc3aee870b /usr/local/MATLAB/R2014a/bin/glnxa64/libmwbridge.so+00145163 _Z8mnParserv+00000587 [122] 0x00007fdc4421f92f /usr/local/MATLAB/R2014a/bin/glnxa64/libmwmcr.so+00489775 _ZN11mcrInstance30mnParser_on_interpreter_threadEv+00000031 [123] 0x00007fdc44200b6d /usr/local/MATLAB/R2014a/bin/glnxa64/libmwmcr.so+00363373 [124] 0x00007fdc44200be9 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwmcr.so+00363497 [125] 0x00007fdc448cfe33 /usr/local/MATLAB/R2014a/bin/glnxa64/libmwservices.so+01646131 _ZN10eventqueue18UserEventQueueImpl5flushEv+00000611 [126] 0x00007fdc388561ed /usr/local/MATLAB/R2014a/bin/glnxa64/libmwuix.so+00221677 [127] 0x00007fdc449756ea /usr/local/MATLAB/R2014a/bin/glnxa64/libmwservices.so+02324202
This error was detected while a MEX-file was running. If the MEX-file is not an official MathWorks function, please examine its source code for errors. Please consult the External Interfaces Guide for information on debugging MEX-files.
If this problem is reproducible, please submit a Service Request via: http://www.mathworks.com/support/contact_us/
A technical support engineer might contact you with further information.
Thank you for your help.** This crash report has been saved to disk as /home/leequens/matlab_crash_dump.4377-1 **
MATLAB is exiting because of fatal error Killed
Depends on the model you want to use. The PSPNet101 trained on Cityscapes requires about 9.6GB memory, while the smaller PSPNet50 trained on ADE20K requires only about 3 GB.
