F1=0.01 with vgg16_gcan-afat-u_imcpt_100_stage1.yaml

[Lin-Yijie]

Dear authors, I have conducted experiments using the AFAT-GCAN model on the IMCPT-100 dataset but encountered random matching results. In contrast, directly running GCAN achieves an F1 score of approximately 66.7.

Below are the details and results:

STATISTIC_STEP: 100
RANDOM_SEED: 321
FP16: False
GCAN:
  FEATURE_CHANNEL: 512
  NODE_FEATURE_DIM: 1024
  NODE_HIDDEN_SIZE: [1024]
  SK_ITER_NUM: 20
  SK_EPSILON: 1e-10
  SK_TAU: 0.05
  CROSS_ITER: False
  CROSS_ITER_NUM: 1
AFA:
  UNIV_SIZE: 100
  K_FACTOR: 50.0
  REG_HIDDEN_FEAT: 8
  REGRESSION: False
  HEAD_NUM: 8
  KQV_DIM: 16
  FF_HIDDEN_DIM: 16
  MS_HIDDEN_DIM: 8
  MS_LAYER1_INIT: 10
  MS_LAYER2_INIT: 10
  MEAN_K: True
  K_GNN_LAYER: 2
  TN_NEURONS: 16
  AFAU: True
IMC_PT_SparseGM:
  ROOT_DIR_NPZ: data/IMC-PT-SparseGM/annotations_100
  MAX_KPT_NUM: 100
Start training...
model on device: cuda:0
----------
lr = 2.00e-03, 2.00e-05

Final obtrain

lr = 3.13e-05, 3.13e-07
Epoch 14   Iteration 100  2.20sample/s Loss=7.2406   Ks_Loss=0.0000   Ks_Error=0.0000  
Epoch 14   Iteration 200  2.15sample/s Loss=6.5199   Ks_Loss=0.0000   Ks_Error=0.0000  
Epoch 14   Iteration 300  2.21sample/s Loss=6.7213   Ks_Loss=0.0000   Ks_Error=0.0000  
Epoch 14   Iteration 400  2.26sample/s Loss=5.9973   Ks_Loss=0.0000   Ks_Error=0.0000  
Epoch 14   Iteration 500  2.28sample/s Loss=6.1740   Ks_Loss=0.0000   Ks_Error=0.0000  
Epoch 14   Iteration 600  2.29sample/s Loss=6.3601   Ks_Loss=0.0000   Ks_Error=0.0000  
Epoch 14   Iteration 700  2.12sample/s Loss=6.3862   Ks_Loss=0.0000   Ks_Error=0.0000  
Epoch 14   Iteration 800  2.14sample/s Loss=6.2037   Ks_Loss=0.0000   Ks_Error=0.0000  
Epoch 14   Iteration 900  2.15sample/s Loss=7.3311   Ks_Loss=0.0000   Ks_Error=0.0000  
Epoch 14   Iteration 1000 2.27sample/s Loss=6.1954   Ks_Loss=0.0000   Ks_Error=0.0000  
Epoch 14   Loss: 6.5130

Start evaluation...
In class reichstag: Ks_Loss=0.0000   Ks_Error=0.0000  
In class sacre_coeur: Ks_Loss=0.0000   Ks_Error=0.0000  
In class st_peters_square: Ks_Loss=0.0000   Ks_Error=0.0000  
Matching accuracy
reichstag: p = 0.0113±0.0176, r = 0.0113±0.0176, f1 = 0.0113±0.0176, cvg = 0.3604
sacre_coeur: p = 0.0145±0.0369, r = 0.0145±0.0369, f1 = 0.0145±0.0369, cvg = 0.0014
st_peters_square: p = 0.0149±0.0211, r = 0.0149±0.0211, f1 = 0.0149±0.0211, cvg = 0.0003
average accuracy: p = 0.0136±0.0252, r = 0.0136±0.0252, f1 = 0.0136±0.0252
gt perm mat cache deleted
Evaluation complete in 26m 49s