SimCLR can not converge on CIFAR10.

[sccbhxc]

I train the model using setting as provided in README.md:

python main_supcon.py --batch_size 1024 \
  --learning_rate 0.5 \
  --temp 0.5 \
  --cosine --syncBN \
  --method SimCLR

The loss in tensorboard log doesn't converge as follows: Step Loss 1 52.37631607055664 2 51.31996154785156 3 50.25985336303711 4 49.499568939208984 5 49.09306335449219 6 48.87476348876953 7 48.573631286621094 8 47.988162994384766 9 47.17831802368164 10 46.756771087646484 11 46.54935073852539 12 46.39372253417969 13 46.233951568603516 14 46.09927749633789 15 45.99102783203125 16 45.91159439086914 17 45.81057357788086 18 45.5342903137207 19 45.24546432495117 20 44.95869827270508 21 44.70873260498047 22 44.49382781982422 23 44.37926483154297 24 44.300750732421875 25 44.23575210571289 26 44.17127227783203 27 44.10074996948242 28 44.01371383666992 29 43.969486236572266 30 43.929256439208984 31 43.905479431152344 32 43.84855651855469 33 43.79642105102539 34 43.77620315551758 35 43.73455810546875 36 43.67094039916992 37 43.634544372558594 38 43.63047409057617 39 43.593658447265625 40 43.54821014404297 41 43.52227020263672 42 43.50453186035156 43 43.482696533203125 44 43.438812255859375 45 43.373321533203125 46 43.3624267578125 47 43.338783264160156 48 43.31117630004883 49 43.26757049560547 50 43.25471115112305 51 43.23273468017578 52 43.24850082397461 53 43.206825256347656 54 43.17913818359375 55 43.155879974365234 56 43.145408630371094 57 43.11878204345703 58 43.12966537475586 59 43.11239242553711 60 43.062042236328125 61 43.04346466064453 62 43.037723541259766 63 43.011138916015625 64 42.994407653808594 65 42.99079513549805 66 42.968414306640625 67 42.97438049316406 68 42.9185905456543 69 42.92195129394531 70 42.92424392700195 71 42.89225769042969 72 42.92551040649414 73 42.88059616088867 74 42.89714813232422 75 42.8614501953125 76 42.8603401184082 77 42.82914352416992 78 42.83755111694336 79 42.825382232666016 80 42.82973098754883 81 42.79133987426758 82 42.790252685546875 83 42.770957946777344 84 42.80356979370117 85 42.773014068603516 86 42.77165603637695 87 42.74653244018555 88 42.75342559814453 89 42.741119384765625 90 42.75617980957031 91 42.72799301147461 92 42.748008728027344 93 42.71396255493164 94 42.69316101074219 95 42.670108795166016 96 42.70542907714844 97 42.694095611572266 98 42.67554473876953 99 42.65801239013672 100 42.67591094970703 101 42.64384460449219 102 42.63705062866211 103 42.627235412597656 104 42.64301681518555 105 42.624053955078125 106 42.62214660644531 107 42.60559844970703 108 42.61735534667969 109 42.62445831298828 110 42.60248947143555 111 42.57283020019531 112 42.58974838256836 113 42.59029006958008 114 42.58818054199219 115 42.552772521972656 116 42.594791412353516 117 42.57866287231445 118 42.55373764038086 119 42.55537033081055 120 42.55070877075195 121 42.525482177734375 122 42.559322357177734 123 42.544517517089844 124 42.52219772338867 125 42.535728454589844 126 42.50553894042969 127 42.51144790649414 128 42.52484130859375 129 42.506187438964844 130 42.502166748046875 131 42.49041748046875 132 42.476890563964844 133 42.50419616699219 134 42.46426773071289 135 42.497535705566406 136 42.466556549072266 137 42.45137023925781 138 42.44359588623047 139 42.46516418457031 140 42.45024871826172 141 42.447364807128906 142 42.456512451171875 143 42.42683029174805 144 42.44383239746094 145 42.42966842651367 146 42.42776870727539 147 42.44707107543945 148 42.43446350097656 149 42.43083953857422 150 42.42207336425781 151 42.42733383178711 152 42.42057418823242 153 42.408233642578125 154 42.40159225463867 155 42.43068313598633 156 42.40238571166992 157 42.40227127075195 158 42.410133361816406 159 42.360538482666016 160 42.39555740356445 161 42.406089782714844 162 42.374088287353516 163 42.37333679199219 164 42.37459182739258 165 42.36992263793945 166 42.35036849975586 167 42.370765686035156 168 42.3534049987793 169 42.34937286376953 170 42.34113693237305 171 42.3719367980957 172 42.356689453125 173 42.3499755859375 174 42.35215759277344 175 42.367679595947266 176 42.37221145629883 177 42.35476303100586 178 42.34419631958008 179 42.33774185180664 180 42.34867477416992 181 42.343505859375 182 42.332584381103516 183 42.331886291503906 184 42.30333709716797 185 42.30495834350586 186 42.31267547607422 187 42.331504821777344 188 42.328800201416016 189 42.31763458251953 190 42.293487548828125 191 42.3109016418457 192 42.3144416809082 193 42.29933166503906 194 42.3012809753418 195 42.286216735839844 196 42.30648422241211 197 42.2964973449707 198 42.31986618041992 199 42.29719161987305 200 42.29789733886719 201 42.28410339355469 202 42.28727722167969 203 42.29293441772461 204 42.26756286621094 205 42.26470184326172 206 42.28857421875 207 42.28956985473633 208 42.278404235839844 209 42.27909469604492 210 42.28044891357422 211 42.249176025390625 212 42.26814270019531 213 42.2680778503418 214 42.25834655761719 215 42.24523162841797 216 42.25312042236328 217 42.26319122314453 218 42.25532150268555 219 42.235530853271484 220 42.23767852783203 221 42.23487091064453 222 42.2308349609375 223 42.2532844543457 224 42.222412109375 225 42.24436950683594 226 42.23404312133789 227 42.257808685302734 228 42.25067138671875 229 42.25416564941406 230 42.225440979003906 231 42.220970153808594 232 42.21751403808594 233 42.22220230102539 234 42.19740295410156 235 42.212947845458984 236 42.22639846801758 237 42.225074768066406 238 42.187904357910156 239 42.21226501464844 240 42.2158317565918 241 42.21928787231445 242 42.211605072021484 243 42.19694137573242 244 42.22465515136719 245 42.210567474365234 246 42.211029052734375 247 42.20466995239258 248 42.19947814941406 249 42.18659591674805 250 42.207908630371094 251 42.18763732910156 252 42.16108703613281 253 42.17030334472656 254 42.16826629638672 255 42.1864128112793 256 42.178138732910156 257 42.207794189453125 258 42.17320251464844 259 42.18295669555664 260 42.18595504760742 261 42.18653869628906 262 42.168556213378906 263 42.16699981689453 264 42.14784240722656 265 42.165992736816406 266 42.15581130981445 267 42.18312454223633 268 42.15531539916992 269 42.155147552490234 270 42.13705062866211 271 42.14425277709961 272 42.15453338623047 273 42.15085220336914 274 42.15422821044922 275 42.166961669921875 276 42.13074493408203 277 42.15336608886719 278 42.150901794433594 279 42.14059066772461 280 42.13984298706055 281 42.13311004638672 282 42.14921951293945 283 42.14160919189453 284 42.12321853637695 285 42.133209228515625 286 42.14018249511719 287 42.146156311035156 288 42.148895263671875 289 42.12226486206055 290 42.119327545166016 291 42.12271499633789 292 42.09735107421875 293 42.13063430786133 294 42.1333122253418 295 42.128719329833984 296 42.13404083251953 297 42.11404037475586 298 42.13532257080078 299 42.13797378540039 300 42.12031555175781 301 42.13229751586914 302 42.1247673034668 303 42.10991287231445 304 42.096988677978516 305 42.09716033935547 306 42.09797668457031 307 42.112796783447266 308 42.09970474243164 309 42.10281753540039 310 42.117523193359375 311 42.091373443603516 312 42.08698654174805 313 42.09238052368164 314 42.100791931152344 315 42.08475112915039 316 42.08399200439453 317 42.09626007080078 318 42.07393264770508 319 42.10096740722656 320 42.07324981689453 321 42.06714630126953 322 42.06916809082031 323 42.104183197021484 324 42.07938003540039 325 42.078514099121094 326 42.08796691894531 327 42.08335494995117 328 42.069602966308594 329 42.06338119506836 330 42.06172180175781 331 42.084617614746094 332 42.05459213256836 333 42.06312942504883 334 42.07081985473633 335 42.05907440185547 336 42.04154968261719 337 42.0722770690918 338 42.04973220825195 339 42.055843353271484 340 42.05887222290039 341 42.05158233642578 342 42.043949127197266 343 42.02438735961914 344 42.013797760009766 345 42.03730773925781 346 42.04892349243164 347 42.04452133178711 348 42.01693344116211 349 42.03768539428711 350 42.035709381103516 351 42.03788757324219 352 42.03336715698242 353 42.02366256713867 354 42.03593826293945 355 42.03465270996094 356 42.01338195800781 357 42.0163459777832 358 42.018856048583984 359 42.02410888671875 360 42.01831817626953 361 42.03579330444336 362 42.02202224731445 363 42.02480697631836 364 42.00835418701172 365 41.999298095703125 366 42.02631759643555 367 42.01470947265625 368 41.99877166748047 369 42.00832748413086 370 42.008548736572266 371 41.99556350708008 372 41.995269775390625 373 42.00551223754883 374 42.00926971435547 375 42.016422271728516 376 41.9869384765625 377 42.00880813598633 378 41.98211669921875 379 42.00701141357422 380 41.989627838134766 381 42.001365661621094 382 41.98550033569336 383 41.99925994873047 384 41.985443115234375 385 41.98691940307617 386 41.964473724365234 387 41.971683502197266 388 41.97077178955078 389 41.96350860595703 390 41.97527313232422 391 41.96897888183594 392 41.96701431274414 393 41.97261047363281 394 41.97776412963867 395 41.993186950683594 396 41.994205474853516 397 41.97821044921875 398 41.95921325683594 399 41.94550323486328 400 41.97230529785156 401 41.95723342895508 402 41.95551300048828 403 41.97821044921875 404 41.96847915649414 405 41.977046966552734 406 41.9465217590332 407 41.95376968383789 408 41.94062423706055 409 41.95761489868164 410 41.93984603881836 411 41.941184997558594 412 41.959529876708984 413 41.95286178588867 414 41.943336486816406 415 41.949058532714844 416 41.925315856933594 417 41.9235725402832 418 41.923301696777344 419 41.94486999511719 420 41.91834259033203 421 41.93389129638672 422 41.94923782348633 423 41.91939163208008 424 41.94850540161133 425 41.94353103637695 426 41.95414733886719 427 41.91848373413086 428 41.93660354614258 429 41.93687438964844 430 41.91402053833008 431 41.91341781616211 432 41.91807174682617 433 41.91950988769531 434 41.9164924621582 435 41.92567443847656 436 41.92313766479492 437 41.92038345336914 438 41.90835952758789 439 41.904605865478516 440 41.900421142578125 441 41.901634216308594 442 41.9135856628418 443 41.892555236816406 444 41.89800262451172 445 41.912933349609375 446 41.9108772277832 447 41.88585662841797 448 41.8737678527832 449 41.90205383300781 450 41.8900032043457 451 41.886817932128906 452 41.902130126953125 453 41.888797760009766 454 41.88259506225586 455 41.874332427978516 456 41.87350082397461 457 41.8810920715332 458 41.887630462646484 459 41.8868522644043 460 41.88219451904297 461 41.8549690246582 462 41.871681213378906 463 41.876739501953125 464 41.86948776245117 465 41.86659622192383 466 41.86448669433594 467 41.84907913208008 468 41.864837646484375 469 41.863677978515625 470 41.8576545715332 471 41.86296081542969 472 41.85995864868164 473 41.87732696533203 474 41.84720993041992 475 41.85145568847656 476 41.85929870605469 477 41.85707092285156 478 41.83163833618164 479 41.82987594604492 480 41.83938980102539 481 41.827880859375 482 41.85377502441406 483 41.820987701416016 484 41.839542388916016 485 41.84109115600586 486 41.82099914550781 487 41.839717864990234 488 41.82582473754883 489 41.83481979370117 490 41.841758728027344 491 41.81592559814453 492 41.82505798339844 493 41.83061599731445 494 41.81941223144531 495 41.838600158691406 496 41.802886962890625 497 41.812198638916016 498 41.815799713134766 499 41.817596435546875 500 41.82366943359375 501 41.83607864379883 502 41.80805206298828 503 41.81154251098633 504 41.80747985839844 505 41.80255126953125 506 41.780853271484375 507 41.786861419677734 508 41.80241775512695 509 41.792362213134766 510 41.79828643798828 511 41.80982208251953 512 41.77899169921875 513 41.79093551635742 514 41.7978401184082 515 41.785980224609375 516 41.773624420166016 517 41.786102294921875 518 41.79802322387695 519 41.782596588134766 520 41.77231979370117 521 41.78274154663086 522 41.78874969482422 523 41.781307220458984 524 41.79595184326172 525 41.781002044677734 526 41.75046920776367 527 41.768497467041016 528 41.771583557128906 529 41.77878952026367 530 41.773895263671875 531 41.75687026977539 532 41.762786865234375 533 41.768733978271484 534 41.74891662597656 535 41.74837875366211 536 41.74883270263672 537 41.75190353393555 538 41.74944305419922 539 41.74364471435547 540 41.77530288696289 541 41.74995803833008 542 41.727783203125 543 41.741119384765625 544 41.73719787597656 545 41.730220794677734 546 41.73965072631836 547 41.75122833251953 548 41.74309158325195 549 41.75099182128906 550 41.72608184814453 551 41.72199249267578 552 41.719600677490234 553 41.731605529785156 554 41.709617614746094 555 41.72447967529297 556 41.73017120361328 557 41.70937728881836 558 41.71445846557617 559 41.7201042175293 560 41.708614349365234 561 41.72523880004883 562 41.71660232543945 563 41.705814361572266 564 41.71420669555664 565 41.725250244140625 566 41.700626373291016 567 41.69807052612305 568 41.69969940185547 569 41.70137023925781 570 41.68778610229492 571 41.68761444091797 572 41.69614791870117 573 41.67936706542969 574 41.67458724975586 575 41.67314529418945 576 41.67515182495117 577 41.67046356201172 578 41.67753601074219 579 41.667606353759766 580 41.67475128173828 581 41.6955451965332 582 41.66904067993164 583 41.670658111572266 584 41.64606857299805 585 41.644012451171875 586 41.668724060058594 587 41.65727233886719 588 41.682044982910156 589 41.66094970703125 590 41.67304611206055 591 41.65132141113281 592 41.6454963684082 593 41.66543197631836 594 41.65175247192383 595 41.67536163330078 596 41.64577102661133 597 41.662147521972656 598 41.644168853759766 599 41.6528205871582 600 41.64643478393555 601 41.63383865356445 602 41.63182830810547 603 41.63913345336914 604 41.6375732421875 605 41.6337890625 606 41.628639221191406 607 41.6407470703125 608 41.636878967285156 609 41.62710952758789 610 41.636619567871094 611 41.610801696777344 612 41.6071662902832 613 41.60697555541992 614 41.626426696777344 615 41.618568420410156 616 41.615962982177734 617 41.59489059448242 618 41.60431671142578 619 41.58615493774414 620 41.608211517333984 621 41.607261657714844 622 41.58834457397461 623 41.619384765625 624 41.589935302734375 625 41.59272003173828 626 41.59053421020508 627 41.58329391479492 628 41.59050750732422 629 41.57986068725586 630 41.58558654785156 631 41.59832000732422 632 41.584468841552734 633 41.58831787109375 634 41.5704345703125 635 41.58049774169922 636 41.573272705078125 637 41.567806243896484 638 41.5692138671875 639 41.56473922729492 640 41.5532341003418 641 41.56444549560547 642 41.562835693359375 643 41.55622100830078 644 41.55236053466797 645 41.559200286865234 646 41.560638427734375 647 41.54945755004883 648 41.55522537231445 649 41.549259185791016 650 41.5435791015625 651 41.54538345336914 652 41.55497741699219 653 41.535770416259766 654 41.5565071105957 655 41.5403938293457 656 41.5374641418457 657 41.52592849731445 658 41.530433654785156 659 41.529991149902344 660 41.53873825073242 661 41.52483367919922 662 41.52631759643555 663 41.52540969848633 664 41.50712203979492 665 41.52194595336914 666 41.508949279785156 667 41.50340270996094 668 41.51433563232422 669 41.506813049316406 670 41.509037017822266 671 41.511940002441406 672 41.48933410644531 673 41.49421691894531 674 41.51228713989258 675 41.48188018798828 676 41.5179328918457 677 41.51668930053711 678 41.4907341003418 679 41.4776611328125 680 41.478702545166016 681 41.4755744934082 682 41.470985412597656 683 41.473052978515625 684 41.46126937866211 685 41.467952728271484 686 41.468509674072266 687 41.458740234375 688 41.46387481689453 689 41.46723175048828 690 41.479740142822266 691 41.451942443847656 692 41.459686279296875 693 41.4549446105957 694 41.44732666015625 695 41.45650863647461 696 41.436283111572266 697 41.4654541015625 698 41.45811080932617 699 41.440711975097656 700 41.448612213134766 701 41.444801330566406 702 41.424259185791016 703 41.42734146118164 704 41.42820739746094 705 41.41264343261719 706 41.428672790527344 707 41.433197021484375 708 41.41559600830078 709 41.42765808105469 710 41.41390609741211 711 41.41726303100586 712 41.41946029663086 713 41.40107345581055 714 41.41954040527344 715 41.40443420410156 716 41.39267349243164 717 41.418861389160156 718 41.411834716796875 719 41.40359115600586 720 41.396331787109375 721 41.390079498291016 722 41.37653732299805 723 41.37510299682617 724 41.387081146240234 725 41.37966537475586 726 41.371559143066406 727 41.3805046081543 728 41.37225341796875 729 41.36929702758789 730 41.36435317993164 731 41.36384963989258 732 41.36091613769531 733 41.362030029296875 734 41.35578918457031 735 41.3519172668457 736 41.35300827026367 737 41.358482360839844 738 41.347862243652344 739 41.37198257446289 740 41.35877990722656 741 41.35511016845703 742 41.33341598510742 743 41.3454704284668 744 41.338096618652344 745 41.32744216918945 746 41.3354606628418 747 41.318241119384766 748 41.33156204223633 749 41.326908111572266 750 41.34585189819336 751 41.32599639892578 752 41.32189178466797 753 41.31526184082031 754 41.3254280090332 755 41.31843185424805 756 41.316680908203125 757 41.31470489501953 758 41.31026840209961 759 41.29541015625 760 41.289337158203125 761 41.2948112487793 762 41.30418395996094 763 41.30305862426758 764 41.288063049316406 765 41.30720520019531 766 41.30964279174805 767 41.28276443481445 768 41.28688049316406 769 41.28529357910156 770 41.27595901489258 771 41.28208923339844 772 41.28265380859375 773 41.27253341674805 774 41.268795013427734 775 41.26056671142578 776 41.26545715332031 777 41.26414108276367 778 41.26213836669922 779 41.254638671875 780 41.264461517333984 781 41.255760192871094 782 41.24617385864258 783 41.253177642822266 784 41.24758529663086 785 41.247886657714844 786 41.24738311767578 787 41.247623443603516 788 41.23225021362305 789 41.23267364501953 790 41.23270034790039 791 41.21902847290039 792 41.21857452392578 793 41.22606658935547 794 41.2296257019043 795 41.21458053588867 796 41.22758483886719 797 41.22265625 798 41.213966369628906 799 41.205604553222656 800 41.20670700073242 801 41.207645416259766 802 41.19804763793945 803 41.19807052612305 804 41.19544982910156 805 41.18919372558594 806 41.19837951660156 807 41.18947982788086 808 41.180660247802734 809 41.189613342285156 810 41.17950439453125 811 41.1776123046875 812 41.18416213989258 813 41.163352966308594 814 41.16533660888672 815 41.16762161254883 816 41.168434143066406 817 41.17054748535156 818 41.1525993347168 819 41.172096252441406 820 41.17144775390625 821 41.151554107666016 822 41.15265655517578 823 41.156288146972656 824 41.14745330810547 825 41.13243865966797 826 41.13278579711914 827 41.12716293334961 828 41.138736724853516 829 41.133758544921875 830 41.11493682861328 831 41.12683868408203 832 41.12635040283203 833 41.133602142333984 834 41.13371658325195 835 41.10513687133789 836 41.10260009765625 837 41.101417541503906 838 41.10969543457031 839 41.11932373046875 840 41.1033821105957 841 41.1043586730957 842 41.095211029052734 843 41.096317291259766 844 41.0890007019043 845 41.09583282470703 846 41.09392547607422 847 41.08848190307617 848 41.0904655456543 849 41.084877014160156 850 41.08682632446289 851 41.0809211730957 852 41.093109130859375 853 41.077423095703125 854 41.0758056640625 855 41.0784797668457 856 41.07194137573242 857 41.07193374633789 858 41.06140899658203 859 41.0634651184082 860 41.05282211303711 861 41.050453186035156 862 41.05570983886719 863 41.05071258544922 864 41.059444427490234 865 41.05086898803711 866 41.054115295410156 867 41.040008544921875 868 41.03927993774414 869 41.044532775878906 870 41.03165817260742 871 41.04341506958008 872 41.03428649902344 873 41.0253791809082 874 41.0325927734375 875 41.015987396240234 876 41.016868591308594 877 41.01715087890625 878 41.01891326904297 879 41.022621154785156 880 41.015464782714844 881 41.012969970703125 882 41.00737762451172 883 41.008392333984375 884 41.01278305053711 885 41.00990295410156 886 41.0096321105957 887 41.007503509521484 888 41.00027084350586 889 40.9947395324707 890 40.99526596069336 891 41.001243591308594 892 41.007118225097656 893 40.98543167114258 894 40.989437103271484 895 40.991146087646484 896 40.98223114013672 897 40.99018478393555 898 40.980106353759766 899 40.98013687133789 900 40.97758483886719 901 40.97883605957031 902 40.97135925292969 903 40.96484375 904 40.97567367553711 905 40.96530532836914 906 40.9644775390625 907 40.9733772277832 908 40.96117401123047 909 40.95415496826172 910 40.96121597290039 911 40.955406188964844 912 40.95452117919922 913 40.95441436767578 914 40.953819274902344 915 40.94948959350586 916 40.94862365722656 917 40.9548454284668 918 40.9460563659668 919 40.95115280151367 920 40.942928314208984 921 40.93732833862305 922 40.94527053833008 923 40.94487762451172 924 40.93137741088867 925 40.931636810302734 926 40.94733428955078 927 40.93010711669922 928 40.9337158203125 929 40.936161041259766 930 40.93152618408203 931 40.932403564453125 932 40.94334411621094 933 40.92143630981445 934 40.9327278137207 935 40.922019958496094 936 40.934322357177734 937 40.92058181762695 938 40.92997741699219 939 40.91799545288086 940 40.92168045043945 941 40.915496826171875 942 40.911827087402344 943 40.91210174560547 944 40.91031265258789 945 40.90728759765625 946 40.909610748291016 947 40.90523910522461 948 40.917598724365234 949 40.90797805786133 950 40.92169189453125 951 40.91349792480469 952 40.90373229980469 953 40.91371536254883 954 40.913360595703125 955 40.91090393066406 956 40.908233642578125 957 40.90796661376953 958 40.90784454345703 959 40.90428924560547 960 40.90233612060547 961 40.89424133300781 962 40.903560638427734 963 40.89834213256836 964 40.890567779541016 965 40.89088821411133 966 40.890174865722656 967 40.896453857421875 968 40.897300720214844 969 40.889373779296875 970 40.89697265625 971 40.89498519897461 972 40.90333557128906 973 40.9049186706543 974 40.88907241821289 975 40.897438049316406 976 40.899349212646484 977 40.88850021362305 978 40.89493179321289 979 40.878849029541016 980 40.886966705322266 981 40.89059829711914 982 40.88986587524414 983 40.90074157714844 984 40.88678741455078 985 40.89237976074219 986 40.888328552246094 987 40.88460922241211 988 40.891021728515625 989 40.893646240234375 990 40.881473541259766 991 40.88935089111328 992 40.88579559326172 993 40.88222885131836 994 40.892173767089844 995 40.882301330566406 996 40.8866081237793 997 40.88020324707031 998 40.88868713378906 999 40.882789611816406 1000 40.88542556762695

[HobbitLong]

I think the loss has been scaled up by 1/0.07. IIRC, I had similar loss, so could you test the model with main_linear.py and see the accuracy?

[sccbhxc]

I think the loss has been scaled up by 1/0.07. IIRC, I had similar loss, so could you test the model with main_linear.py and see the accuracy?

I test the trained model with main_linear.py. The final accuracy is only 14.98%.

[HobbitLong]

Do you use the same setting as mine?

[sccbhxc]

Do you use the same setting as mine?

Yes

[jizongFox]

For me I had something near 93% @sccbhxc

[sccbhxc]

For me I had something near 93% @sccbhxc

Thank you, I retry it.

[tarunn2799]

Same issue, I'm currently training SimCLR on the default settings given by you, and my loss has constantly been approx 30.00 (± 2) from epoch 1 to 90. What could be the issue? You're saying despite it being like this here, it'll change when I run my linear eval?

[HobbitLong]

@tarunn2799, have you tried to adjust the learning rate?

[tarunn2799]

@HobbitLong Yes, I've tried adjusting it between 0.1 to 1, the loss still is the same from epoch 1 to 100. Infact I'm just directly running the same code on the repository with no changes except disabling syncbn. What could it possibly be?

[tsaishien-chen]

For me I had something near 93% @sccbhxc

@jizongFox hihi, could you provide the log of training loss (or the final converge of loss) for your 93% accuracy model?

[LefanZhang]

@sccbhxc I encountered the same problem. Have you solved it?

[yuhui-zh15]

I encountered the same problem when trying to use syncbn without apex support. Just removing --syncbn solves the problem.