FileNotFoundError: [Errno 2] No such file or directory: 'data/json_feat_2.1.0/tests_seen/trial_T20190909_042500_949430/traj_data.json'

[TopCoder2K]

I follow the instructions to install and run FILM. After running python models/train/train_seq2seq.py --data data/json_feat_2.1.0 --model seq2seq_im_mask --dout exp/model:{model},name:pm_and_subgoals_01 --splits data/splits/oct21.json --gpu --batch 8 --pm_aux_loss_wt 0.1 --subgoal_aux_loss_wt 0.1 --preprocess I get an error:

Namespace(action_loss_wt=1.0, actor_dropout=0.0, attn_dropout=0.0, batch=8, data='data/json_feat_2.1.0', dataset_fraction=0, dec_teacher_forcing=False, decay_epoch=10, demb=100, dframe=2500, dhid=512, dout='exp/model:seq2seq_im_mask,name:pm_and_subgoals_01', epoch=20, fast_epoch=False, gpu=True, hstate_dropout=0.3, input_dropout=0.0, lang_dropout=0.0, lr=0.0001, mask_loss_wt=1.0, model='seq2seq_im_mask', pframe=300, pm_aux_loss_wt=0.1, pp_folder='pp', preprocess=True, resume=None, save_every_epoch=False, seed=123, splits='data/splits/oct21.json', subgoal_aux_loss_wt=0.1, temp_no_history=False, use_templated_goals=False, vis_dropout=0.3, zero_goal=False, zero_instr=False)
{'tests_seen': 1533,
 'tests_unseen': 1529,
 'train': 21023,
 'valid_seen': 820,
 'valid_unseen': 821}

Preprocessing dataset and saving to pp folders ... This will take a while. Do this once as required.
Preprocessing tests_seen
  0% (0 of 1533) |                                                                                                                                                  | Elapsed Time: 0:00:00 ETA:  --:--:--Traceback (most recent call last):
  File "models/train/train_seq2seq.py", line 88, in <module>
    dataset.preprocess_splits(splits)
  File "/home/svyatoslav/Internship/EQA/alfred/data/preprocess.py", line 54, in preprocess_splits
    with open(json_path) as f:
FileNotFoundError: [Errno 2] No such file or directory: 'data/json_feat_2.1.0/tests_seen/trial_T20190909_042500_949430/traj_data.json'
100% (1533 of 1533) |###############################################################################################################################################| Elapsed Time: 0:00:00 Time:  0:00:00

Actually, there is data/json_2.1.0/tests_seen/trial_T20190909_042500_949430 directory, not data/json_feat_2.1.0/tests_seen/trial_T20190909_042500_949430. So, it seems the correct command is

python models/train/train_seq2seq.py --data data/json_2.1.0 --model seq2seq_im_mask --dout exp/model:{model},name:pm_and_subgoals_01 --splits data/splits/oct21.json --gpu --batch 8 --pm_aux_loss_wt 0.1 --subgoal_aux_loss_wt 0.1 --preprocess

UPD from 05/12/2022

Both variants are correct, but for those who don't want to download the Modeling Quickstart dataset it's easier to use just the Lite version, because images are not used in the preprocessing. Please correct me if I'm wrong, @soyeonm. Or maybe it should be mentioned that "You need the Modeling Quickstart dataset" so that people don't encounter the error in the title.

[TopCoder2K]

I also want to note that best_model_multi.pt is extracted as best_model_multi.pt and new_best_model.pt in the instructions: It looks like a typo, since new_best_model.pt is also present in the the .zip archive.

P. S. I do not think it's worth opening a separate issue, that's why I pointed it out here.