py-faster-rcnn-OHEM icon indicating copy to clipboard operation
py-faster-rcnn-OHEM copied to clipboard

fast_rcnn_ohem train, bbox loss always be 'nan'

Open pobizheV opened this issue 8 years ago • 3 comments

Thinks for your sharing. I try to use this faster rcnn to deal with my own data. However, when I train stage of fast_rcnn, there is a problem that bbox loss always be 'nan'. Could you please give me some help. Thank you! I0426 19:55:49.662811 30135 solver.cpp:229] Iteration 0, loss = nan I0426 19:55:49.662861 30135 solver.cpp:245] Train net output #0: loss_bbox = nan (* 1 = nan loss) I0426 19:55:49.662870 30135 solver.cpp:245] Train net output #1: loss_cls = 1.88031 (* 1 = 1.88031 loss) I0426 19:55:49.662895 30135 sgd_solver.cpp:106] Iteration 0, lr = 0.001 I0426 19:55:58.697458 30135 solver.cpp:229] Iteration 20, loss = nan I0426 19:55:58.697528 30135 solver.cpp:245] Train net output #0: loss_bbox = nan (* 1 = nan loss) I0426 19:55:58.697536 30135 solver.cpp:245] Train net output #1: loss_cls = 0.0256684 (* 1 = 0.0256684 loss) I0426 19:55:58.697545 30135 sgd_solver.cpp:106] Iteration 20, lr = 0.001 I0426 19:56:07.779089 30135 solver.cpp:229] Iteration 40, loss = nan I0426 19:56:07.779152 30135 solver.cpp:245] Train net output #0: loss_bbox = nan (* 1 = nan loss) I0426 19:56:07.779160 30135 solver.cpp:245] Train net output #1: loss_cls = 0.0125189 (* 1 = 0.0125189 loss)

pobizheV avatar Apr 26 '17 12:04 pobizheV

maybe learning rate is too large

makefile avatar Nov 02 '17 09:11 makefile

@pobizheV Have you solved the issue? I also met the problem.

blueardour avatar Jan 15 '18 00:01 blueardour

You can check the box gt if there is 0 or negative in some case

| | 邹进屹 邮箱:[email protected] |

签名由 网易邮箱大师 定制

On 01/15/2018 08:44, blueardour wrote:

@pobizheV Have you solved the issue? I also met the problem.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

manutdzou avatar Jan 16 '18 14:01 manutdzou