train at mpii,acc so small

[mathpopo]

I try to run "python example/mpii.py -a hg --stacks 8 --blocks 1 --checkpoint checkpoint/hg_s8_b1/ " in your code , compare to your log.txt,loss can descend as same with you,but acc samll too mush,just "python example/mpii.py -a hg --stacks 8 --blocks 1 --checkpoint checkpoint/hg_s8_b1/ " is whole?

[yuanyuanli85]

looks like the train_loss does not drop too much. Have you follow the instruction to disable cudnn for batchnorm layer if you are using pytorch0.4.x?

[mathpopo]

i use titan xp ,can run well,this effect is in the v100,the two graphics cards are different?

[mathpopo]

disable cudnn for batchnorm layer: sed -i "1194s/torch.backends.cudnn.enabled/False/g"?i do this already,use pytorch 0.4.1,cuda 9.0

[mathpopo]

can you tell me your graphics cards hardware?

[mathpopo]

i use v100,resume from pre-trained model ,acc descend ,use titan xp , go up

[yuanyuanli85]

The gpu I used is titan xp, not v100. I guess some low-level difference b/w driver/cudnn caused the problem in v100.

[mathpopo]

thank you very mush.sorry ,other issue,if half body & side ,effect so bad,i descend threshold ,point error happen

[mathpopo]

this is .0.1 threshold effect