nan value

[Gavinic]

when i try to train a thai model, there are some nan/inf accrued in DeformableTransformer. and i check the out put of self.detection_transformer: the output is [nan, nan, nan,......] so i go to deeper: i have found that the problem is the forward of 'DeformableTransformer', but i don know how to fix it.

my config file is blew: BASE: "../Base_det.yaml"

MODEL: WEIGHTS: "" TRANSFORMER: VOC_SIZE: 230 NUM_POINTS: 50

DATASETS: TRAIN: ("thaicard_train",) TEST: ("thaicard_eval",)

SOLVER: IMS_PER_BATCH: 4 BASE_LR: 1e-4 LR_BACKBONE: 1e-4 WARMUP_ITERS: 0 STEPS: (320000,) MAX_ITER: 375000 CHECKPOINT_PERIOD: 100000

TEST: EVAL_PERIOD: 10000

OUTPUT_DIR: "output/vitaev2_s/thai_pre"

How to fix it ? Thank you!

[Gavinic]

here is my train.json file. thank you!

[xiaoming-master]

Modify configuration file parameters：MODEL.TRANSFORMER.VOC_SIZE: [yourself voc size]