MetaFormer
MetaFormer copied to clipboard
dqshuai
about acc of stanfordcars
good work, I achieve perforemance of CUB-200-2011(92.3%), but I only got 93.73% on stanfordcars. Is it any other different configure compared with CUB-200-2011 except for lr?
In my experiment, lr had the greatest impact on the experimental results. Try --lr 5e-3 --min-lr 5e-7 --warmup-lr 5e-8 --epochs 300 --warmup-epochs 20 with batchsize 512?
In my experiment, lr had the greatest impact on the experimental results. Try
--lr 5e-3 --min-lr 5e-7 --warmup-lr 5e-8 --epochs 300 --warmup-epochs 20?
python3 -m torch.distributed.launch --nproc_per_node 2 --master_port 12335 main.py --cfg ./configs/MetaFG_1_224.yaml --batch-size 32 --tag stcar_v2 --lr 5e-3 --min-lr 5e-7 --warmup-lr 5e-8 --epochs 300 --warmup-epochs 20 --dataset stanfordcars --pretrain ./pretrained_model/metafg_1_inat21_384.pth --accumulation-steps 8 --opts DATA.IMG_SIZE 384. This is my config. I did use the recommended lr. Is it other different with cub-200-2011?
In my experiment, lr had the greatest impact on the experimental results. Try
--lr 5e-3 --min-lr 5e-7 --warmup-lr 5e-8 --epochs 300 --warmup-epochs 20?
python3 -m torch.distributed.launch --nproc_per_node 2 --master_port 12335 main.py --cfg ./configs/MetaFG_1_224.yaml --batch-size 32 --tag stcar_v2 --lr 5e-3 --min-lr 5e-7 --warmup-lr 5e-8 --epochs 300 --warmup-epochs 20 --dataset stanfordcars --pretrain ./pretrained_model/metafg_1_inat21_384.pth --accumulation-steps 8 --opts DATA.IMG_SIZE 384.This is my config. I did use the recommended lr. Is it other different with cub-200-2011?
The actual training lr is related to batchsize, specifically actual_lr = lr * total_batchsize/512
In my experiment, lr had the greatest impact on the experimental results. Try
--lr 5e-3 --min-lr 5e-7 --warmup-lr 5e-8 --epochs 300 --warmup-epochs 20?
python3 -m torch.distributed.launch --nproc_per_node 2 --master_port 12335 main.py --cfg ./configs/MetaFG_1_224.yaml --batch-size 32 --tag stcar_v2 --lr 5e-3 --min-lr 5e-7 --warmup-lr 5e-8 --epochs 300 --warmup-epochs 20 --dataset stanfordcars --pretrain ./pretrained_model/metafg_1_inat21_384.pth --accumulation-steps 8 --opts DATA.IMG_SIZE 384.This is my config. I did use the recommended lr. Is it other different with cub-200-2011?The actual training lr is related to batchsize, specifically actual_lr = lr * total_batchsize/512
Recommanded config includes --nproc_per_node 8 --batch-size 32 --accumulation-steps 2. There are only 2*3090 for me, So I modify these into --nproc_per_node 2 --batch-size 32 --accumulation-steps 8 to maintain same total_batchsize. Is it right? I also notad ``fused_weight_gradient_mlp_cuda module not found. gradient accumulation fusion with weight gradient computation disabled. in my log file. Dose it mean that --accumulation-steps dose not work in my code?