Hyper parameter for cifar10-vggsmall

[BaptisteNguyen]

Hello, What are the hyper parameter for the training of vggsmall on cifar 10?

[haibao-yu]

Hello, What are the hyper parameter for the training of vggsmall on cifar 10?

Hi, how is the problem going? I also implement lsq method with resnet20 on cifar, but there is more than 3% Top-1 Accuracy drop.

[haibao-yu]

Hello, What are the hyper parameter for the training of vggsmall on cifar 10?

Hi, how is the problem going? I also implement lsq method with resnet20 on cifar, but there is more than 3% Top-1 Accuracy drop.

For ResNet20 on Cifar10, I have gotten Top-1 Accuracy 90.2% with 2bit for weight and 2bit for activation, compared to 91.8% with full precision. I think the results are quite good, which will be better with further hyperparameters tuning. Following are the experiment details:

1. to train the full precision as a pre-trained model: I set the initial learning rate as 0.1 with Cosine schedule and 160 epochs, weight decay as 1e-4, batch size as 128
2. to initialize the step size: I set 1.0 for activation as the README
3. to train the quantized model: I set the initial learning rate as 0.2 with Cosine schedule and 90 epochs, weight decay as 1e-4, batch size as 512

[BaptisteNguyen]

Hello, Thank you for your answer. I will test this.

        Baptiste Nguyen

---

De : walk2out [[email protected]] Envoyé : samedi 9 janvier 2021 12:58 À : hustzxd/LSQuantization Cc : BaptisteNguyen; Author Objet : Re: [hustzxd/LSQuantization] Hyper parameter for cifar10-vggsmall (#2)

Hello, What are the hyper parameter for the training of vggsmall on cifar 10?

Hi, how is the problem going? I also implement lsq method with resnet20 on cifar, but there is more than 3% Top-1 Accuracy drop.

For ResNet20 on Cifar10, I have gotten Top-1 Accuracy 90.2% with 2bit for weight and 2bit for activation, compared to 91.8% with full precision. I think the results are quite good, which will be better with further hyperparameters tuning. Following are the experiment details:

1. to train the full precision as a pre-trained model: I set the initial learning rate as 0.1 with Cosine schedule and 160 epochs, weight decay as 1e-4, batch size as 128
2. to initialize the step size: I set 1.0 for activation as the README
3. to train the quantized model: I set the initial learning rate as 0.2 with Cosine schedule and 90 epochs, weight decay as 1e-4, batch size as 512

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/hustzxd/LSQuantization/issues/2#issuecomment-757138917, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQEMV62KVRAL3FRVNVZMUATSZBAHNANCNFSM4TW3UMQQ.