layout-model-training icon indicating copy to clipboard operation
layout-model-training copied to clipboard

Predicted boxes or scores contain Inf/NaN. Training has diverged.

Open MachineLearning4Work opened this issue 5 months ago • 1 comments

I encountered an issue while running the training code using train_net.py. The error suggests a FloatingPointError in the file "proposal_utils.py" at line 103, indicating that predicted boxes or scores contain Inf/NaN, leading to training divergence.

File "/usr/local/lib/python3.10/dist-packages/detectron2/modeling/proposal_generator/proposal_utils.py", line 103, in find_top_rpn_proposals

raise FloatingPointError(

FloatingPointError: Predicted boxes or scores contain Inf/NaN. Training has diverged.

How can I resolve this issue?

MachineLearning4Work avatar Jan 24 '24 18:01 MachineLearning4Work