missing_aware_prompts icon indicating copy to clipboard operation
missing_aware_prompts copied to clipboard

Published 20 hours ago • verified publisher icon YiLunLee

About experiment setting

Open jhyukjang opened this issue 1 year ago • 5 comments

Thx for your great work! I have some questions about your code.

  1. Did you set the random seed to 0 for all experiments?
  2. I reproduced as you did (NUM_GPUS=2, BS_FITS_YOUR_GPU=2) like below

ARROW_ROOT=./datasets/mmimdb NUM_GPUS=2 NUM_NODES=1 BS_FITS_YOUR_GPU=2 PRETRAINED_MODEL_PATH=./pretrained_weight/vilt_200k_mlm_itm.ckpt EXP_NAME=mmimdb

python run.py with data_root=${ARROW_ROOT}
num_gpus=${NUM_GPUS}
num_nodes=${NUM_NODES}
per_gpu_batchsize=${BS_FITS_YOUR_GPU}
task_finetune_mmimdb
load_path=${PRETRAINED_MODEL_PATH}
exp_name=${EXP_NAME}

and I got 40.65 (paper: 42.66) on test set with same setting. Can I reproduce the paper's results without changing parameters like a learning rate or are there some optimized hyperparameters for each dataset?

jhyukjang avatar May 23 '23 04:05 jhyukjang