Error stay very high before 100 epoch

[RocStone]

I simply download your code and run cifar100.py on K80 with torch == 1.1.0 torchvision == 0.2.2 But the trian and test error is still high even having been trained by 100 epochs. Did I do anything wrong?

Following is the output of terminal Files already downloaded and verified Files already downloaded and verified ======== epoch 0 (lr: 0.01600) ======== train loss = 4.75042 | train err = 98.99% | test loss = 4.60554 | test err = 99.01% ======== epoch 1 (lr: 0.01600) ======== train loss = 4.60711 | train err = 99.05% | test loss = 4.60681 | test err = 99.01% ======== epoch 2 (lr: 0.01552) ======== train loss = 4.60656 | train err = 98.97% | test loss = 4.60671 | test err = 99.01% ======== epoch 3 (lr: 0.01552) ======== train loss = 4.60680 | train err = 99.07% | test loss = 4.60664 | test err = 99.01% ======== epoch 4 (lr: 0.01505) ======== train loss = 4.60665 | train err = 98.95% | test loss = 4.60659 | test err = 99.01% ======== epoch 5 (lr: 0.01505) ======== train loss = 4.60670 | train err = 98.98% | test loss = 4.60672 | test err = 99.01% ======== epoch 6 (lr: 0.01505) ======== train loss = 4.60649 | train err = 99.08% | test loss = 4.60659 | test err = 99.01% ======== epoch 7 (lr: 0.01460) ======== train loss = 4.60658 | train err = 99.04% | test loss = 4.60645 | test err = 99.01% ======== epoch 8 (lr: 0.01460) ======== train loss = 4.60652 | train err = 99.03% | test loss = 4.60651 | test err = 98.94% ======== epoch 9 (lr: 0.01416) ======== train loss = 4.60657 | train err = 99.11% | test loss = 4.60653 | test err = 99.01% ======== epoch 10 (lr: 0.01416) ======== train loss = 4.60649 | train err = 98.97% | test loss = 4.60653 | test err = 99.01% ======== epoch 11 (lr: 0.01374) ======== train loss = 4.60658 | train err = 98.96% | test loss = 4.60603 | test err = 99.01% ======== epoch 12 (lr: 0.01374) ======== train loss = 4.60644 | train err = 99.00% | test loss = 4.60641 | test err = 99.01% ======== epoch 13 (lr: 0.01374) ======== train loss = 4.60625 | train err = 99.00% | test loss = 4.60647 | test err = 99.01% ======== epoch 14 (lr: 0.01333) ======== train loss = 4.60643 | train err = 99.04% | test loss = 4.60629 | test err = 99.01% ======== epoch 15 (lr: 0.01333) ======== train loss = 4.60619 | train err = 99.02% | test loss = 4.60632 | test err = 99.01% ======== epoch 16 (lr: 0.01293) ======== train loss = 4.60616 | train err = 99.04% | test loss = 4.60624 | test err = 99.01% ======== epoch 17 (lr: 0.01293) ======== train loss = 4.60627 | train err = 99.00% | test loss = 4.60632 | test err = 99.01% ======== epoch 18 (lr: 0.01293) ======== train loss = 4.60628 | train err = 98.98% | test loss = 4.60630 | test err = 99.01% ======== epoch 19 (lr: 0.01254) ======== train loss = 4.60628 | train err = 99.00% | test loss = 4.60613 | test err = 99.01% ======== epoch 20 (lr: 0.01254) ======== train loss = 4.60622 | train err = 98.94% | test loss = 4.60623 | test err = 99.01% ======== epoch 21 (lr: 0.01216) ======== train loss = 4.60622 | train err = 99.05% | test loss = 4.60608 | test err = 99.01% ======== epoch 22 (lr: 0.01216) ======== train loss = 4.60613 | train err = 98.93% | test loss = 4.60622 | test err = 99.01% ======== epoch 23 (lr: 0.01180) ======== train loss = 4.60618 | train err = 98.95% | test loss = 4.60577 | test err = 99.01% ======== epoch 24 (lr: 0.01180) ======== train loss = 4.60615 | train err = 99.02% | test loss = 4.60619 | test err = 99.01% ======== epoch 25 (lr: 0.01180) ======== train loss = 4.60616 | train err = 99.00% | test loss = 4.60606 | test err = 99.01% ======== epoch 26 (lr: 0.01144) ======== train loss = 4.60617 | train err = 99.04% | test loss = 4.60606 | test err = 99.01% ======== epoch 27 (lr: 0.01144) ======== train loss = 4.60593 | train err = 98.94% | test loss = 4.60610 | test err = 99.01% ======== epoch 28 (lr: 0.01110) ======== train loss = 4.60595 | train err = 99.03% | test loss = 4.60600 | test err = 99.01% ======== epoch 29 (lr: 0.01110) ======== train loss = 4.60597 | train err = 99.02% | test loss = 4.60595 | test err = 99.01% ======== epoch 30 (lr: 0.01110) ======== train loss = 4.60599 | train err = 98.91% | test loss = 4.60603 | test err = 99.01% ======== epoch 31 (lr: 0.01077) ======== train loss = 4.60592 | train err = 99.00% | test loss = 4.60591 | test err = 98.94% ======== epoch 32 (lr: 0.01077) ======== train loss = 4.60596 | train err = 99.00% | test loss = 4.60596 | test err = 99.01% ======== epoch 33 (lr: 0.01045) ======== train loss = 4.60598 | train err = 98.92% | test loss = 4.60590 | test err = 99.01% ======== epoch 34 (lr: 0.01045) ======== train loss = 4.60598 | train err = 98.89% | test loss = 4.60590 | test err = 99.01% ======== epoch 35 (lr: 0.01013) ======== train loss = 4.60594 | train err = 99.03% | test loss = 4.60560 | test err = 99.01% ======== epoch 36 (lr: 0.01013) ======== train loss = 4.60595 | train err = 99.08% | test loss = 4.60580 | test err = 98.94% ======== epoch 37 (lr: 0.01013) ======== train loss = 4.60583 | train err = 98.92% | test loss = 4.60591 | test err = 99.01% ======== epoch 38 (lr: 0.00983) ======== train loss = 4.60586 | train err = 99.00% | test loss = 4.60580 | test err = 98.94% ======== epoch 39 (lr: 0.00983) ======== train loss = 4.60583 | train err = 99.02% | test loss = 4.60581 | test err = 99.01% ======== epoch 40 (lr: 0.00953) ======== train loss = 4.60594 | train err = 98.94% | test loss = 4.60580 | test err = 99.01% ======== epoch 41 (lr: 0.00953) ======== train loss = 4.60585 | train err = 99.07% | test loss = 4.60586 | test err = 99.01% ======== epoch 42 (lr: 0.00953) ======== train loss = 4.60593 | train err = 99.06% | test loss = 4.60571 | test err = 99.01% ======== epoch 43 (lr: 0.00925) ======== train loss = 4.60587 | train err = 98.96% | test loss = 4.60572 | test err = 99.01% ======== epoch 44 (lr: 0.00925) ======== train loss = 4.60573 | train err = 98.91% | test loss = 4.60576 | test err = 98.94% ======== epoch 45 (lr: 0.00897) ======== train loss = 4.60579 | train err = 99.09% | test loss = 4.60572 | test err = 99.01% ======== epoch 46 (lr: 0.00897) ======== train loss = 4.60568 | train err = 99.05% | test loss = 4.60574 | test err = 99.01% ======== epoch 47 (lr: 0.00870) ======== train loss = 4.60585 | train err = 99.04% | test loss = 4.60555 | test err = 99.01% ======== epoch 48 (lr: 0.00870) ======== train loss = 4.60557 | train err = 98.98% | test loss = 4.60565 | test err = 98.94% ======== epoch 49 (lr: 0.00870) ======== train loss = 4.60570 | train err = 98.96% | test loss = 4.60565 | test err = 98.94% ======== epoch 50 (lr: 0.00844) ======== train loss = 4.60566 | train err = 99.02% | test loss = 4.60564 | test err = 99.01% ======== epoch 51 (lr: 0.00844) ======== train loss = 4.60564 | train err = 99.01% | test loss = 4.60572 | test err = 99.01% ======== epoch 52 (lr: 0.00819) ======== train loss = 4.60576 | train err = 99.01% | test loss = 4.60564 | test err = 99.01% ======== epoch 53 (lr: 0.00819) ======== train loss = 4.60566 | train err = 99.07% | test loss = 4.60567 | test err = 99.01% ======== epoch 54 (lr: 0.00819) ======== train loss = 4.60566 | train err = 98.99% | test loss = 4.60572 | test err = 99.01% ======== epoch 55 (lr: 0.00794) ======== train loss = 4.60581 | train err = 99.03% | test loss = 4.60558 | test err = 98.94% ======== epoch 56 (lr: 0.00794) ======== train loss = 4.60555 | train err = 98.98% | test loss = 4.60566 | test err = 99.01% ======== epoch 57 (lr: 0.00770) ======== train loss = 4.60555 | train err = 99.00% | test loss = 4.60560 | test err = 99.01% ======== epoch 58 (lr: 0.00770) ======== train loss = 4.60575 | train err = 99.04% | test loss = 4.60568 | test err = 99.01% ======== epoch 59 (lr: 0.00747) ======== train loss = 4.60564 | train err = 99.01% | test loss = 4.60546 | test err = 99.01% ======== epoch 60 (lr: 0.00747) ======== train loss = 4.60566 | train err = 99.06% | test loss = 4.60559 | test err = 99.01% ======== epoch 61 (lr: 0.00747) ======== train loss = 4.60559 | train err = 98.99% | test loss = 4.60558 | test err = 99.01% ======== epoch 62 (lr: 0.00725) ======== train loss = 4.60551 | train err = 99.06% | test loss = 4.60557 | test err = 99.01% ======== epoch 63 (lr: 0.00725) ======== train loss = 4.60566 | train err = 98.99% | test loss = 4.60555 | test err = 99.01% ======== epoch 64 (lr: 0.00703) ======== train loss = 4.60563 | train err = 99.03% | test loss = 4.60562 | test err = 99.01% ======== epoch 65 (lr: 0.00703) ======== train loss = 4.60559 | train err = 99.04% | test loss = 4.60551 | test err = 98.94% ======== epoch 66 (lr: 0.00703) ======== train loss = 4.60555 | train err = 99.02% | test loss = 4.60549 | test err = 99.01% ======== epoch 67 (lr: 0.00682) ======== train loss = 4.60547 | train err = 99.04% | test loss = 4.60552 | test err = 99.01% ======== epoch 68 (lr: 0.00682) ======== train loss = 4.60565 | train err = 99.06% | test loss = 4.60559 | test err = 99.01% ======== epoch 69 (lr: 0.00661) ======== train loss = 4.60551 | train err = 98.94% | test loss = 4.60550 | test err = 98.94% ======== epoch 70 (lr: 0.00661) ======== train loss = 4.60546 | train err = 98.97% | test loss = 4.60552 | test err = 99.01% ======== epoch 71 (lr: 0.00642) ======== train loss = 4.60546 | train err = 98.97% | test loss = 4.60539 | test err = 99.01% ======== epoch 72 (lr: 0.00642) ======== train loss = 4.60555 | train err = 99.07% | test loss = 4.60548 | test err = 99.01% ======== epoch 73 (lr: 0.00642) ======== train loss = 4.60542 | train err = 98.97% | test loss = 4.60550 | test err = 99.01% ======== epoch 74 (lr: 0.00622) ======== train loss = 4.60550 | train err = 98.92% | test loss = 4.60546 | test err = 99.01% ======== epoch 75 (lr: 0.00622) ======== train loss = 4.60547 | train err = 98.98% | test loss = 4.60545 | test err = 99.01% ======== epoch 76 (lr: 0.00604) ======== train loss = 4.60542 | train err = 99.03% | test loss = 4.60539 | test err = 99.01% ======== epoch 77 (lr: 0.00604) ======== train loss = 4.60546 | train err = 98.98% | test loss = 4.60546 | test err = 99.01% ======== epoch 78 (lr: 0.00604) ======== train loss = 4.60552 | train err = 99.06% | test loss = 4.60551 | test err = 99.01% ======== epoch 79 (lr: 0.00586) ======== train loss = 4.60542 | train err = 99.00% | test loss = 4.60548 | test err = 99.01% ======== epoch 80 (lr: 0.00586) ======== train loss = 4.60553 | train err = 99.01% | test loss = 4.60543 | test err = 99.01% ======== epoch 81 (lr: 0.00568) ======== train loss = 4.60548 | train err = 99.00% | test loss = 4.60545 | test err = 99.01% ======== epoch 82 (lr: 0.00568) ======== train loss = 4.60537 | train err = 98.94% | test loss = 4.60541 | test err = 99.01% ======== epoch 83 (lr: 0.00551) ======== train loss = 4.60549 | train err = 98.97% | test loss = 4.60538 | test err = 99.01% ======== epoch 84 (lr: 0.00551) ======== train loss = 4.60545 | train err = 99.09% | test loss = 4.60545 | test err = 99.01% ======== epoch 85 (lr: 0.00551) ======== train loss = 4.60553 | train err = 99.07% | test loss = 4.60543 | test err = 99.01% ======== epoch 86 (lr: 0.00534) ======== train loss = 4.60541 | train err = 98.98% | test loss = 4.60543 | test err = 99.01% ======== epoch 87 (lr: 0.00534) ======== train loss = 4.60542 | train err = 99.03% | test loss = 4.60543 | test err = 99.01% ======== epoch 88 (lr: 0.00518) ======== train loss = 4.60545 | train err = 98.98% | test loss = 4.60537 | test err = 99.01% ======== epoch 89 (lr: 0.00518) ======== train loss = 4.60553 | train err = 98.91% | test loss = 4.60538 | test err = 99.01% ======== epoch 90 (lr: 0.00518) ======== train loss = 4.60535 | train err = 98.96% | test loss = 4.60542 | test err = 99.01% ======== epoch 91 (lr: 0.00503) ======== train loss = 4.60539 | train err = 98.98% | test loss = 4.60541 | test err = 99.01% ======== epoch 92 (lr: 0.00503) ======== train loss = 4.60538 | train err = 99.01% | test loss = 4.60545 | test err = 99.01% ======== epoch 93 (lr: 0.00488) ======== train loss = 4.60539 | train err = 99.02% | test loss = 4.60539 | test err = 99.01% ======== epoch 94 (lr: 0.00488) ======== train loss = 4.60539 | train err = 99.01% | test loss = 4.60533 | test err = 99.01% ======== epoch 95 (lr: 0.00473) ======== train loss = 4.60539 | train err = 98.97% | test loss = 4.60533 | test err = 99.01% ======== epoch 96 (lr: 0.00473) ======== train loss = 4.60532 | train err = 98.95% | test loss = 4.60539 | test err = 99.01% ======== epoch 97 (lr: 0.00473) ======== train loss = 4.60537 | train err = 99.05% | test loss = 4.60535 | test err = 98.94% ======== epoch 98 (lr: 0.00459) ======== train loss = 4.60541 | train err = 99.00% | test loss = 4.60537 | test err = 99.01% ======== epoch 99 (lr: 0.00459) ======== train loss = 4.60544 | train err = 98.99% | test loss = 4.60538 | test err = 99.01% ======== epoch 100 (lr: 0.00445) ======== train loss = 4.60533 | train err = 98.96% | test loss = 4.60536 | test err = 99.01% ======== epoch 101 (lr: 0.00445) ======== train loss = 4.60529 | train err = 98.93% | test loss = 4.60533 | test err = 99.01% ======== epoch 102 (lr: 0.00445) ======== train loss = 4.60531 | train err = 98.98% | test loss = 4.60536 | test err = 99.01% ======== epoch 103 (lr: 0.00432) ======== train loss = 4.60539 | train err = 99.12% | test loss = 4.60530 | test err = 98.94%

[GitGhidorah]

I dont know well. but i think you will get better result, by changing the following. for example,(by using this optimizer)

optimizer = torch.optim.SGD(model.parameters(), lr=0.01, momentum=0.9, weight_decay=1e-5)

i think you will see quickly on the 1st epoch.

[rgd66]

hello i have some questions .i can't run the cifar100.py becuase swish.py have some trouble.can you tell me what should i do ?