badge
badge copied to clipboard
JordanAsh
Got different accuracy results
Thank you for sharing the source code for this great work.
I am trying to replicate some of your baseline results. However, the result from the baseline (LeastConfidence) I got is much better than the one reported in your paper. Here is my result accuracy plot (It achieves 40% test accuracy much earlier than the reported accuracy in the paper).
Query size: 100
Dataset: CIFAR-10
Network: Resnet
Strategy: LeastConfidence
Should I do something to replicate your results?
Yes, the default learning rate isn't correct. It should be an order of magnitude lower, I believe.
On Wed, Jul 29, 2020 at 7:21 PM Sungwon Han [email protected] wrote:
Thank you for sharing the source code for this great work.
I am trying to replicate some of your baseline results. However, the result from the baseline (LeastConfidence) I got is much better than the one reported in your paper. Here is my result accuracy plot (It achieves 40% test accuracy much earlier than the reported accuracy in the paper).
Query size: 100 Dataset: CIFAR-10 Network: Resnet Strategy: LeastConfidence [image: image] https://user-images.githubusercontent.com/17738624/88863172-25f6f100-d23d-11ea-9933-c0848eb44727.png
Should I do something to replicate your results?
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/JordanAsh/badge/issues/1, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEAIDG6QH7DZA46YMCWZVFLR6CVFXANCNFSM4PMI6ZDA .
Even after changing the learning rate as same as reported in the paper, I can't replicate the paper's result. Are there any other things that I should do for replication?
Right, I'm saying that I believe the learning rate in the paper is off by an order of magnitude.
On Wed, Jul 29, 2020 at 10:56 PM Sungwon Han [email protected] wrote:
Even after changing the learning rate as same as reported in the paper, I can't replicate the paper's result. Are there any other things that I should do for replication?
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/JordanAsh/badge/issues/1#issuecomment-666058643, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEAIDG7TY6TI6CNYMIEK2QDR6DOO7ANCNFSM4PMI6ZDA .
Then, can you provide a command to replicate the paper's result? Or, are there any seeds to replicate the paper's result?