Variations-of-SFANet-for-Crowd-Counting icon indicating copy to clipboard operation
Variations-of-SFANet-for-Crowd-Counting copied to clipboard

Problems when training

Open KrystalCWT opened this issue 3 years ago • 3 comments

Hi, I trained M-SFANet with part of shanghaiTech samples. BUT the loss converge too slow, and the mse and mae remain large after training a few hundred epoches. Do you know why?

KrystalCWT avatar Jan 19 '21 02:01 KrystalCWT

If you have employed the Bayesian preprocessing, the convergence could be a bit slow. Sometimes it can take up to 600-800 epochs to converge. I think you can first try the Gaussian filter with fixed std to see the performance. And also, try experimenting with look ahead optimizer like in this paper, https://arxiv.org/abs/1907.08610, to enhance convergence rate.

Pongpisit-Thanasutives avatar Jan 22 '21 08:01 Pongpisit-Thanasutives

Hi, I trained M-SFANet with part of shanghaiTech samples. BUT the loss converge too slow, and the mse and mae remain large after training a few hundred epoches. Do you know why?

How many epochs did you train? And how much MAE value you get (large)?

phapnm avatar Nov 10 '21 03:11 phapnm

@phapnm Based on my experience, using either the Bayesian preprocessing or Gaussian filter with fixed std, If training up to >700 (700-1000) epochs, the model should converge on SHA (MAE<60) and SHB (MAE<7). To train 1000 epochs, it may take 1-2 days on a single GPU.

I am planning to release the preprocessed dataset of SH and UCF-QNRF for better reproducibility.

Thanks!

Pongpisit-Thanasutives avatar Nov 10 '21 04:11 Pongpisit-Thanasutives