Ze Liu

Hi @EnricoBeltramo, thanks for pointing this. We have fixed it https://github.com/microsoft/Swin-Transformer/pull/228. If you want to use Swin-MoE, please refer to [get_started.md#mixture-of-experts-support](https://github.com/microsoft/Swin-Transformer/blob/78cec9ac5c746c7e72305a9a24716ddb3fcc043c/get_started.md?plain=1#L231-L246) for more instructions, if not, please just ignore it.

Hi @zllljf, this means the **total-batch-size** is 4096. **total-batch-size** = **batch-size-per-gpu** x **#gpu**

Hi @crazypatient0, here is a similar issue https://github.com/microsoft/Swin-Transformer/issues/158. Hope it can help.

Instructions and configs for fine-tuning on higher resolution can be found here: https://github.com/microsoft/Swin-Transformer/blob/main/get_started.md#fine-tuning-on-higher-resolution

Hi @choasup, you are right, but the flops of `avgpool` is so negligible compared with the overall flops, so we just ignore it.

Hi @lcmeng, training logs are available here: https://github.com/microsoft/Swin-Transformer/blob/b05e6214a37d33846903585c9e83b694ef411587/README.md?plain=1#L79-L81

The logs of Swin-S/B were generated by an earlier version of the code, which didn't write the configs. However, the configs are the same with the configs provided [here](https://github.com/microsoft/Swin-Transformer/blob/main/README.md#main-results-on-imagenet-with-pretrained-models). 16...

Hi @he-y, the total batch size should be 1024, but yours is 512. Try this command: ``` CUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch --nproc_per_node 4 --master_port 12345 main.py --cfg configs/swin_base_patch4_window7_224.yaml --data-path /dataset/imagenet...

Hi @lcmeng, the model is trained using the default **mixed-precision (O1)**. We doesn't deal with the logging of amp so the loss scaling info is not wrote to the log...

The code can be found here: https://github.com/microsoft/Swin-Transformer/blob/b05e6214a37d33846903585c9e83b694ef411587/models/swin_transformer.py#L578-L585