CurveNet icon indicating copy to clipboard operation
CurveNet copied to clipboard

about random_seed in voting strategy

Open Roywangj opened this issue 3 years ago • 4 comments

hi, i write a voting_eval python program. It loads the trained model and test for M times, however, the result of everytime time is the same. I change the random seed but failed. Would you please open your voting_eval codes? Thanks!

Roywangj avatar Nov 23 '21 12:11 Roywangj

Hi, the voting script we used was identical to the ones used in RSCNN, therefore we didn't keep particular records of the script itself. It is really weird that the voting results are the same every time. Can you please make sure:

  1. the averaged results (rather than the argmax index) indeed change across different trails?
  2. the inputs are indeed different for each vote? inputs should be translated or jittered randomly at each vote. Since we are essentially relying on randomness for different augmentations on the input data, and eventually, different results, the random seed is preferred not to be specified. Please keep me informed of this weird behavior!

tiangexiang avatar Nov 23 '21 14:11 tiangexiang

i use different seed and max result. Indeed, I think 'farthest = torch.randint(0, N, (B,), dtype=torch.long).to(device) * 0' is wrong. (in models/curvenet_utils.py). Could you please check the codes

Roywangj avatar Nov 23 '21 14:11 Roywangj

This is because, in FPS, we only want to start from the first index node (0th node) for eliminating any randomness. If this causes the problem in voting, maybe you can try to get rid of the trailing `* 0'.

tiangexiang avatar Nov 23 '21 14:11 tiangexiang

In addition, i failed to translate or jitter input, it indeed output different results, but results are very bad(about 5%). I check the RS-CNN code, i find they use the same "new_fps_idx" in training and testing process, while curvenet doesn't use it in training, but i use it in testing. Would you please tell me how you set the translate or jitter process in detail?

Roywangj avatar Nov 23 '21 14:11 Roywangj