Validation accuracy on CLEVR

[aelnouby]

Hi,

I have been trying to implement the relational network paper train it on CLEVR dataset, however my results are way off the results reported in the paper. I am curious to know what was the best results you got using your code(train/validation) ?

Another question, in the paper and in your code it is stated that the final layer has a 29 neurons corresponding to 29 classes. However when I parsed and tokenized the answers they were only 28 unique answers, what is the 29th class ?

Thanks.

[rosinality]

I got about 65% test accuracy. (I remember train accuracy was about the same.) When I use 2x2 pooling in the convolution module then I got about 75% train accuracy and 55% test accuracy.

And number of the classes is 28. I think it is typo and I just used that number.

[aelnouby]

Thanks so much for your reply.

[xeniaqian94]

Hi @rosinality , thanks for sharing this awesome implementation. Wondering if this issue has been resolved, i.e. test accuracy goes over 90% now. Thanks!

[rosinality]

@xeniaqian94 Yes, I got about 92-93% by using tricks suggested by @mesnico. It is crucial to use a kind of warm up learning rate schedules.

[xeniaqian94]

Thanks Kim for this nice confirmation!

On May 2, 2019, at 9:03 PM, Kim Seonghyeon [email protected] wrote:

@xeniaqian94 https://github.com/xeniaqian94 Yes, I got about 92-93% by using tricks suggested by @mesnico https://github.com/mesnico. It is crucial to use a kind of warm up learning rate schedules.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/rosinality/relation-networks-pytorch/issues/1#issuecomment-488883261, or mute the thread https://github.com/notifications/unsubscribe-auth/AB3ASH72ZMGL2M4QUB4S4FTPTOFN7ANCNFSM4E2PHITA.