the segformer not converge on my custom dataset

[AncientRemember]

custom dataset

1. foreground is very samll,hair line
2. consist some noisy labels/missed label
3. foreground/background is very imbalance,99% is background

experiments

i has try many times to train with hrnet-ocr,and the performance is Robust when i trying segformer,at beginng the model looks like converging,but after 10000 iters,the performance begin go bad,and finaly the acc ,mIou end up with zero

init_cfg=dict(type='Pretrained', checkpoint='pretrained/segformer_mit-b0.pth'),
batchsize = 16
optimizer = dict(
    type='AdamW',
    lr=0.00006,
    betas=(0.9, 0.999),
    weight_decay=0.01,
    paramwise_cfg=dict(
        custom_keys={
            'pos_block': dict(decay_mult=0.),
            'norm': dict(decay_mult=0.),
            'head': dict(lr_mult=10.)
        }))
optimizer_config = dict()

lr_config = dict(
    policy='poly',
    warmup='linear',
    warmup_iters=1500,
    warmup_ratio=1e-6,
    power=1.0,
    min_lr=0.0,
    by_epoch=False)
runner = dict(type='IterBasedRunner', max_iters=160000)
checkpoint_config = dict(by_epoch=False, interval=10000)
evaluation = dict(interval=10000, metric='mIoU')

i will try to decrease the lr to 0.00001

other

i change the attention module's parameter,add need_weight=False,because the pytorch has an issue with tensor.sum(),and we really not need weightsum

[Amr-Mustafa]

Did you reach a stable configuration?

[QuietWoods]

What is the value of num_classes?