perception loss and performance

[Mollylulu]

Hi~ thanks for your work! I am trying to train your model. Following your readme, I first try to train the perception module. However, during training, I meet nan in loss problems.

I wonder did you meet this issue in your training? and how do you solve it?

for accomplish training, I skip this nan iteration. But I find the loss is quite high and increases after 10k steps. (however in your code, you are using 100 epochs for training. )

Looking forward to your response. Thanks!

[Ahnhojin1223]

I have same issue when I train perception module near 40 epoch (57974 step)