Coefficient lambda values not provided anywhere in the research paper or issues

[Dayan-Zhanchi]

trafficstars

I can't seem to find the values of the coefficient lambdas of the loss functions. Could you please provide these values? They don't appear in the research paper or any of the issues as far as I know, I've even looked at issue #14, but it's not there.

Also I got some follow-up questions from my issue #33 regarding the consumer-commercial clothing item retrieval:

1. Specifically what is the evaluation of the chosen detected consumer clothing item based on? Is it based on the quote below? And so if the result of the proposed method in the below quote is positive, does that mean that the evaluation of the chosen detected consumer clothing item will be positive?

If IoU between retrieved item from shop images and one of the ground truth corresponding gallery item is over the thresh(we set thresh as 0.5), the retrieved result is positive.

2. I still don't understand fully how the gallery item is retrieved? The matching network only outputs a similarity score, it doesn't retrieve anything. So do you compare the similarity score of ALL gallery items with the chosen detected consumer clothing item, and choose the gallery item with the highest similarity score?

3. Also where are the gallery items in the validation dataset? Do you have to construct them? And do the gallery set contain ALL the commerical clothing items?

4. Since you used the top-k accuracy metric for clothing retrieval, does that mean that you have to choose k gallery items with the highest similarity score for a given chosen detected consumer clothing item?

[xwjabc]

+1 Hope that the author can provide the coefficient of the losses.

[Dayan-Zhanchi]

@geyuying

If the coefficient lambda values are sensitive information (for business purposes) can you as the author at least confirm that you can't give this information out? This will be important for my group since we are kind of conducting a reproducibility project, but don't have unlimited GPU resources to find the optimal values through too many experiments. If we know that the coefficient lambda values can't be released, then we can at least mention that in our report.

[xwjabc]

@Dayan-Zhanchi I reproduced the match-net baseline with mask features (i.e. use the RoI features after the same RoIAlign with mask branch's). My re-implementation is based on maskrcnn-benchmark. Reported results in https://codalab.lri.fr/competitions/565#learn_the_details-evaluation top-1/top-10/top-20: 0.135/0.350/0.447 My reproduced results: top-1/top-10/top-20: 0.1637/0.3604/0.4479 I am still validating the results and training the model for 2nd trial. Currently, I use 2 x TITAN RTX GPUs and it takes ~33 hrs to train. Some details (e.g. #epochs / loss coefficients / sampling strategy) may be different from the original paper.

[simaoh]

@xwjabc Can you explain the value lambdas you used for each of the different losses?

[xwjabc]

@xwjabc Can you explain the value lambdas you used for each of the different losses?

Hi simaoh, I simply use 1 as the coefficient for all losses.

[joppichristian]

@xwjabc Could you share with me the number of epochs, batch size and lr optimizer steps? I also use two 2 x TITAN RTX GPU. Thank you!

[xwjabc]

@xwjabc Could you share with me the number of epochs, batch size and lr optimizer steps? I also use two 2 x TITAN RTX GPU. Thank you!

Some information of one experiment I conducted is shown below:

[DONE] 2x Titan RTX, 8 images per GPU, paper setting PYTHONPATH=./ python -m torch.distributed.launch --nproc_per_node=2 ./tools/train_net.py --config-file "configs/deepfashion2/e2e_match_rcnn_R_50_FPN_1x_paper.yaml" SOLVER.IMS_PER_BATCH 16 SOLVER.BASE_LR 0.02 SOLVER.MAX_ITER 90000 SOLVER.STEPS "(60000, 80000)" TEST.IMS_PER_BATCH 16 MODEL.RPN.FPN_POST_NMS_TOP_N_TRAIN 8000 OUTPUT_DIR "../data/deepfashion2_match_paper_2gpu" (test on 2-GPU, 4 images per GPU, final_model, multi gpus eval) PYTHONPATH=./ python -m torch.distributed.launch --nproc_per_node=2 ./tools/test_net.py --config-file "configs/deepfashion2/e2e_match_rcnn_R_50_FPN_1x_paper.yaml" --ckpt "../data/deepfashion2_match_paper_2gpu/model_final.pth" TEST.IMS_PER_BATCH 8 MODEL.RPN.FPN_POST_NMS_TOP_N_TRAIN 4000 OUTPUT_DIR "../data/deepfashion2_match_paper_2gpu/inference_final" MODEL.MASK_ON False TEST.MULTI_GPU_EVAL True Results: 0.1637/0.2828/0.3604/0.4107/0.4479

In this experiment, I use 2x Titan RTX and put 8 images per GPU (that is, batch size = 16). For epochs and optimizations, I use the following settings in the YAML config file from maskrcnn-benchmark:

SOLVER:
  BASE_LR: 0.02
  WEIGHT_DECAY: 0.0001
  STEPS: (60000, 80000)
  MAX_ITER: 90000

[joppichristian]

@xwjabc Thanks a lot!