unlabel_pred error and cannot find the images

[Luojlong]

batch_mlvl_bboxes /= batch_mlvl_bboxes.new_tensor( [>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 5142/5142, 19.1 task/s, elapsed: 269s, ETA: 0s2022-11-24 10:22:08,173 - mmdet - INFO - [INFO] Unlabel pred Done! Traceback (most recent call last): File "tools/train.py", line 202, in main() File "tools/train.py", line 190, in main train_detector( File "/home/hello/PycharmProjects/pythonProject/new_DSL/DSL/mmdet/apis/train.py", line 218, in train_detector runner.run(data_loaders, cfg.workflow) File "/home/hello/PycharmProjects/pythonProject/new_DSL/DSL/mmdet/runner/hooks/semi_epoch_based_runner.py", line 345, in run epoch_runner(data_loaders[i], **kwargs) File "/home/hello/PycharmProjects/pythonProject/new_DSL/DSL/mmdet/runner/hooks/semi_epoch_based_runner.py", line 267, in train self.call_hook('after_train_iter') File "/home/hello/anaconda3/envs/Torch-DSL/lib/python3.8/site-packages/mmcv/runner/base_runner.py", line 307, in call_hook getattr(hook, fn_name)(self) File "/home/hello/PycharmProjects/pythonProject/new_DSL/DSL/mmdet/runner/hooks/unlabel_pred_hook.py", line 460, in after_train_iter self.after_train_iter_func(runner) File "/home/hello/PycharmProjects/pythonProject/new_DSL/DSL/mmdet/runner/hooks/unlabel_pred_hook.py", line 517, in after_train_iter_func assert len(runner.imagefiles) == 2 AssertionError ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 16269) of binary: /home/hello/anaconda3/envs/Torch-DSL/bin/python

when i want to debug and set preload=1,start_point=2 to reduce training time.It occur another error.

loading annotations into memory... Done (t=0.00s) creating index... index created! loading annotations into memory... Done (t=0.13s) creating index... index created! loading annotations into memory... Done (t=0.02s) creating index... index created! loading annotations into memory... Done (t=0.20s) creating index... index created! [ERROR][ModelInfer] Found no image in /home/hello/PycharmProjects/pythonProject/new_DSL/DSL/mydata/semicoco/images/full

but this document has images,i dont know what happen? anybody can help me?Thanks a lot.

[chenbinghui1]

@progincline please double check the image path；and set preload=2*num_worker+2 when per_gpu_image_num = 2

[Luojlong]

图片路径是没有问题的，num_worker是指workers_per_gpu的值吗，如果是我就没有设错。但问题是只要运行就会报[ERROR][ModelInfer] Found no image in /home/hello/PycharmProjects/pythonProject/new_DSL/DSL/mydata/semicoco/images/full，我昨天也遇到这个问题了，然后重新再配了一遍代码就没事了，但报错后再运行就会报这个错，所以现在有点困惑。

[Luojlong]

我把train，unlabel_train/pred，val的图片路径都在.sh设好了，就是有点不清楚这个报错是哪个部分出来的

[chenbinghui1]

@progincline 我看你用的是自己的数据，所以不太清楚是否生成了对应的semicoco的格式文件， 例如ann_file 这些，可以再check下，或者debug下，打印一下对应位置的信息；