The network spends too much time on some images

[Ethanwl]

When I test my trained model on the test data of ShanghaiTechA, there are 92 images, each of which takes more than 1 s and meanwhile there are 61 images, each of which takes less than 0.1 s. My GPU is GTX 1070. My test method is

torch.cuda.synchronize()
start = time.time()
density_map = net(im_data)
density_map = density_map.data.cpu().numpy()
torch.cuda.synchronize()
end =  time.time()

Why does it happen? How can I solve it? Thank you!

[siyuhuang]

@Ethanwl

I haven't seen the problem before. Please check and provide more details.