Question about train on DepthTrack dataset

[TonikLeung]

Hello, I tried to train the DeT DiMP50 Mean on DepthTrack dataset, I set the path and selected a few sequences to testing the training. When I ran “python run_training.py dimp DeT_DiMP50_Mean”, the error raised.

I tracked the error, it was in ltr/trainers/base_trainers.py:

Restarting training from last epoch ... /home/cat/ljt/DeT/checkpoints/ltr/dimp/DeT_DiMP50_Mean/DiMPnet_DeT_ep*.pth.tar Training crashed at epoch 51 Traceback for the error! Traceback (most recent call last): File "../ltr/trainers/base_trainer.py", line 70, in train self.train_epoch() File "../ltr/trainers/ltr_trainer.py", line 80, in train_epoch self.cycle_dataset(loader) File "../ltr/trainers/ltr_trainer.py", line 52, in cycle_dataset for i, data in enumerate(loader, 1): File "/home/cat/anaconda3/envs/DeT/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 521, in next data = self._next_data() File "/home/cat/anaconda3/envs/DeT/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1203, in _next_data return self._process_data(data) File "/home/cat/anaconda3/envs/DeT/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1229, in _process_data data.reraise() File "/home/cat/anaconda3/envs/DeT/lib/python3.7/site-packages/torch/_utils.py", line 425, in reraise raise self.exc_type(msg) ValueError: Caught ValueError in DataLoader worker process 0. Original Traceback (most recent call last): File "pandas/_libs/parsers.pyx", line 1095, in pandas._libs.parsers.TextReader._convert_tokens TypeError: Cannot cast array data from dtype('O') to dtype('float32') according to the rule 'safe'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/cat/anaconda3/envs/DeT/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop data = fetcher.fetch(index) File "/home/cat/anaconda3/envs/DeT/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/cat/anaconda3/envs/DeT/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in data = [self.dataset[idx] for idx in possibly_batched_index] File "../ltr/data/sampler.py", line 108, in getitem seq_info_dict = dataset.get_sequence_info(seq_id) File "../ltr/dataset/depthtrack.py", line 125, in get_sequence_info bbox = self._read_bb_anno(depth_path) File "../ltr/dataset/depthtrack.py", line 95, in _read_bb_anno gt = pandas.read_csv(bb_anno_file, delimiter=',', header=None, dtype=np.float32, na_filter=False, low_memory=False).values File "/home/cat/anaconda3/envs/DeT/lib/python3.7/site-packages/pandas/util/_decorators.py", line 311, in wrapper return func(*args, **kwargs) File "/home/cat/anaconda3/envs/DeT/lib/python3.7/site-packages/pandas/io/parsers/readers.py", line 586, in read_csv return _read(filepath_or_buffer, kwds) File "/home/cat/anaconda3/envs/DeT/lib/python3.7/site-packages/pandas/io/parsers/readers.py", line 488, in _read return parser.read(nrows) File "/home/cat/anaconda3/envs/DeT/lib/python3.7/site-packages/pandas/io/parsers/readers.py", line 1047, in read index, columns, col_dict = self._engine.read(nrows) File "/home/cat/anaconda3/envs/DeT/lib/python3.7/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 229, in read data = self._reader.read(nrows) File "pandas/_libs/parsers.pyx", line 783, in pandas._libs.parsers.TextReader.read File "pandas/_libs/parsers.pyx", line 880, in pandas._libs.parsers.TextReader._read_rows File "pandas/_libs/parsers.pyx", line 1026, in pandas._libs.parsers.TextReader._convert_column_data File "pandas/_libs/parsers.pyx", line 1103, in pandas._libs.parsers.TextReader._convert_tokens ValueError: cannot safely convert passed user dtype of float32 for object dtyped data in column 0

[Yifanpan1]

I meet the same problem，Have you solved it？

[TonikLeung]

Haven't yet..

[JiaLianjie]

I meet the same problem，Have you solved it？

Haven't yet..

I meet the problem too, have you solved it? And can you share your pandas version?

[JiaLianjie]

I meet the same problem，Have you solved it？

Haven't yet..

I have solved it! The matter is caused by a parameter of function "pandas.read_csv()" in file "DeT-main/ltr/dataset/depthtrack.py". Here is the explanation of three parameters in the document:

na_values: scalar, str, list-like, or dict, optional Additional strings to recognize as NA/NaN. If dict passed, specific per-column NA values. By default the following values are interpreted as NaN: ‘’, ‘#N/A’, ‘#N/A N/A’, ‘#NA’, ‘-1.#IND’, ‘-1.#QNAN’, ‘-NaN’, ‘-nan’, ‘1.#IND’, ‘1.#QNAN’, ‘<NA>’, ‘N/A’, ‘NA’, ‘NULL’, ‘NaN’, ‘None’, ‘n/a’, ‘nan’, ‘null’.

keep_default_na: bool, default True Whether or not to include the default NaN values when parsing the data. Depending on whether na_values is passed in, the behavior is as follows:

• If keep_default_na is True, and na_values are specified, na_values is appended to the default NaN values used for parsing.
• If keep_default_na is True, and na_values are not specified, only the default NaN values are used for parsing.
• If keep_default_na is False, and na_values are specified, only the NaN values specified na_values are used for parsing.
• If keep_default_na is False, and na_values are not specified, no strings will be parsed as NaN.

Note that if na_filter is passed in as False, the keep_default_na and na_values parameters will be ignored.

na_filter: bool, default True Detect missing value markers (empty strings and the value of na_values). In data without any NAs, passing na_filter=False can improve the performance of reading a large file.

In depthtrack.py, we can find "na_values=False" in line 95, which blocked the other two parameters so that "read_csv" can not identify "nan" in "groundtruth.txt" as NA values. We just switch it to "True" and it works.

[MOCUISHLE-AC]

Hello, I try to run DeT_ATOM_Max. But I meet some problems when the File "det/ltr/dataset/depthtrack.py" runs 'frame_list = [self._get_frame(depth_path, f_id) for ii, f_id in enumerate(frame_ids)]'(in line 185). It looks like there are some problems in DepthTrack dataset(toy07_indoor_320)

[train: 1, 1 / 1000] FPS: 7.4 (7.4) , Loss/total: 1.04415 , Loss/iou: 1.04415 [train: 1, 2 / 1000] FPS: 13.2 (59.7) , Loss/total: 0.95544 , Loss/iou: 0.95544 [train: 1, 3 / 1000] FPS: 14.5 (18.3) , Loss/total: 1.01399 , Loss/iou: 1.01399 [train: 1, 4 / 1000] FPS: 15.5 (19.0) , Loss/total: 1.00854 , Loss/iou: 1.00854 [train: 1, 5 / 1000] FPS: 16.1 (19.1) , Loss/total: 0.97844 , Loss/iou: 0.97844 [train: 1, 6 / 1000] FPS: 16.5 (19.2) , Loss/total: 0.88426 , Loss/iou: 0.88426 [train: 1, 7 / 1000] FPS: 16.9 (19.3) , Loss/total: 0.82712 , Loss/iou: 0.82712 [train: 1, 8 / 1000] FPS: 16.8 (16.3) , Loss/total: 0.78724 , Loss/iou: 0.78724 [train: 1, 9 / 1000] FPS: 16.9 (18.0) , Loss/total: 0.75835 , Loss/iou: 0.75835 [train: 1, 10 / 1000] FPS: 17.1 (19.2) , Loss/total: 0.73059 , Loss/iou: 0.73059 [train: 1, 11 / 1000] FPS: 17.2 (18.2) , Loss/total: 0.71331 , Loss/iou: 0.71331 [train: 1, 12 / 1000] FPS: 17.4 (19.8) , Loss/total: 0.69950 , Loss/iou: 0.69950 [train: 1, 13 / 1000] FPS: 17.5 (18.9) , Loss/total: 0.67931 , Loss/iou: 0.67931 [train: 1, 14 / 1000] FPS: 17.6 (18.6) , Loss/total: 0.65817 , Loss/iou: 0.65817 [train: 1, 15 / 1000] FPS: 17.6 (18.0) , Loss/total: 0.64252 , Loss/iou: 0.64252 [train: 1, 16 / 1000] FPS: 17.7 (19.6) , Loss/total: 0.62998 , Loss/iou: 0.62998 [train: 1, 17 / 1000] FPS: 17.8 (19.2) , Loss/total: 0.61434 , Loss/iou: 0.61434 [train: 1, 18 / 1000] FPS: 17.8 (18.4) , Loss/total: 0.59853 , Loss/iou: 0.59853 [train: 1, 19 / 1000] FPS: 17.9 (19.3) , Loss/total: 0.58180 , Loss/iou: 0.58180 [train: 1, 20 / 1000] FPS: 18.0 (19.0) , Loss/total: 0.56429 , Loss/iou: 0.56429 [ WARN:[email protected]] global loadsave.cpp:248 findDecoder imread_('/root/siton-gpfs-archive/leiheao/dataset/DepthTrack/train/toy07_indoor_320/color/00001385.jpg'): can't open/read file: check file path/integrity [train: 1, 21 / 1000] FPS: 18.0 (18.8) , Loss/total: 0.55770 , Loss/iou: 0.55770 Training crashed at epoch 1 Traceback for the error! Traceback (most recent call last): File "/root/siton-gpfs-archive/leiheao/det/ltr/trainers/base_trainer.py", line 70, in train self.train_epoch() File "/root/siton-gpfs-archive/leiheao/det/ltr/trainers/ltr_trainer.py", line 82, in train_epoch self.cycle_dataset(loader) File "/root/siton-gpfs-archive/leiheao/det/ltr/trainers/ltr_trainer.py", line 53, in cycle_dataset for i, data in enumerate(loader, 1): File "/opt/conda/envs/lha-env/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 530, in next data = self._next_data() File "/opt/conda/envs/lha-env/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1224, in _next_data return self._process_data(data) File "/opt/conda/envs/lha-env/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1250, in _process_data data.reraise() File "/opt/conda/envs/lha-env/lib/python3.7/site-packages/torch/_utils.py", line 457, in reraise raise exception cv2.error: Caught error in DataLoader worker process 0. Original Traceback (most recent call last): File "/opt/conda/envs/lha-env/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop data = fetcher.fetch(index) File "/opt/conda/envs/lha-env/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/opt/conda/envs/lha-env/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/root/siton-gpfs-archive/leiheao/det/ltr/data/sampler.py", line 161, in getitem train_frames, train_anno, meta_obj_train = dataset.get_frames(seq_id, train_frame_ids, seq_info_dict) File "/root/siton-gpfs-archive/leiheao/det/ltr/dataset/depthtrack.py", line 185, in get_frames frame_list = [self._get_frame(depth_path, f_id) for ii, f_id in enumerate(frame_ids)] File "/root/siton-gpfs-archive/leiheao/det/ltr/dataset/depthtrack.py", line 185, in frame_list = [self._get_frame(depth_path, f_id) for ii, f_id in enumerate(frame_ids)] File "/root/siton-gpfs-archive/leiheao/det/ltr/dataset/depthtrack.py", line 158, in _get_frame img = get_rgbd_frame(color_path, depth_path, dtype=self.dtype, depth_clip=True) File "/root/siton-gpfs-archive/leiheao/det/ltr/dataset/depth_utils.py", line 15, in get_rgbd_frame rgb = cv2.cvtColor(rgb, cv2.COLOR_BGR2RGB) cv2.error: OpenCV(4.8.1) /io/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'

• dataset/DepthTrack/train/toy07_indoor_320/color/00001385.jpg is not exits! thank you if you could help me

[JiaLianjie]

Hello, I try to run DeT_ATOM_Max. But I meet some problems when the File "det/ltr/dataset/depthtrack.py" runs 'frame_list = [self._get_frame(depth_path, f_id) for ii, f_id in enumerate(frame_ids)]'(in line 185). It looks like there are some problems in DepthTrack dataset(toy07_indoor_320)

[train: 1, 1 / 1000] FPS: 7.4 (7.4) , Loss/total: 1.04415 , Loss/iou: 1.04415 [train: 1, 2 / 1000] FPS: 13.2 (59.7) , Loss/total: 0.95544 , Loss/iou: 0.95544 [train: 1, 3 / 1000] FPS: 14.5 (18.3) , Loss/total: 1.01399 , Loss/iou: 1.01399 [train: 1, 4 / 1000] FPS: 15.5 (19.0) , Loss/total: 1.00854 , Loss/iou: 1.00854 [train: 1, 5 / 1000] FPS: 16.1 (19.1) , Loss/total: 0.97844 , Loss/iou: 0.97844 [train: 1, 6 / 1000] FPS: 16.5 (19.2) , Loss/total: 0.88426 , Loss/iou: 0.88426 [train: 1, 7 / 1000] FPS: 16.9 (19.3) , Loss/total: 0.82712 , Loss/iou: 0.82712 [train: 1, 8 / 1000] FPS: 16.8 (16.3) , Loss/total: 0.78724 , Loss/iou: 0.78724 [train: 1, 9 / 1000] FPS: 16.9 (18.0) , Loss/total: 0.75835 , Loss/iou: 0.75835 [train: 1, 10 / 1000] FPS: 17.1 (19.2) , Loss/total: 0.73059 , Loss/iou: 0.73059 [train: 1, 11 / 1000] FPS: 17.2 (18.2) , Loss/total: 0.71331 , Loss/iou: 0.71331 [train: 1, 12 / 1000] FPS: 17.4 (19.8) , Loss/total: 0.69950 , Loss/iou: 0.69950 [train: 1, 13 / 1000] FPS: 17.5 (18.9) , Loss/total: 0.67931 , Loss/iou: 0.67931 [train: 1, 14 / 1000] FPS: 17.6 (18.6) , Loss/total: 0.65817 , Loss/iou: 0.65817 [train: 1, 15 / 1000] FPS: 17.6 (18.0) , Loss/total: 0.64252 , Loss/iou: 0.64252 [train: 1, 16 / 1000] FPS: 17.7 (19.6) , Loss/total: 0.62998 , Loss/iou: 0.62998 [train: 1, 17 / 1000] FPS: 17.8 (19.2) , Loss/total: 0.61434 , Loss/iou: 0.61434 [train: 1, 18 / 1000] FPS: 17.8 (18.4) , Loss/total: 0.59853 , Loss/iou: 0.59853 [train: 1, 19 / 1000] FPS: 17.9 (19.3) , Loss/total: 0.58180 , Loss/iou: 0.58180 [train: 1, 20 / 1000] FPS: 18.0 (19.0) , Loss/total: 0.56429 , Loss/iou: 0.56429 [ WARN:[email protected]] global loadsave.cpp:248 findDecoder imread_('/root/siton-gpfs-archive/leiheao/dataset/DepthTrack/train/toy07_indoor_320/color/00001385.jpg'): can't open/read file: check file path/integrity [train: 1, 21 / 1000] FPS: 18.0 (18.8) , Loss/total: 0.55770 , Loss/iou: 0.55770 Training crashed at epoch 1 Traceback for the error! Traceback (most recent call last): File "/root/siton-gpfs-archive/leiheao/det/ltr/trainers/base_trainer.py", line 70, in train self.train_epoch() File "/root/siton-gpfs-archive/leiheao/det/ltr/trainers/ltr_trainer.py", line 82, in train_epoch self.cycle_dataset(loader) File "/root/siton-gpfs-archive/leiheao/det/ltr/trainers/ltr_trainer.py", line 53, in cycle_dataset for i, data in enumerate(loader, 1): File "/opt/conda/envs/lha-env/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 530, in next data = self._next_data() File "/opt/conda/envs/lha-env/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1224, in _next_data return self._process_data(data) File "/opt/conda/envs/lha-env/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1250, in _process_data data.reraise() File "/opt/conda/envs/lha-env/lib/python3.7/site-packages/torch/_utils.py", line 457, in reraise raise exception cv2.error: Caught error in DataLoader worker process 0. Original Traceback (most recent call last): File "/opt/conda/envs/lha-env/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop data = fetcher.fetch(index) File "/opt/conda/envs/lha-env/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/opt/conda/envs/lha-env/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 49, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/root/siton-gpfs-archive/leiheao/det/ltr/data/sampler.py", line 161, in getitem train_frames, train_anno, meta_obj_train = dataset.get_frames(seq_id, train_frame_ids, seq_info_dict) File "/root/siton-gpfs-archive/leiheao/det/ltr/dataset/depthtrack.py", line 185, in get_frames frame_list = [self._get_frame(depth_path, f_id) for ii, f_id in enumerate(frame_ids)] File "/root/siton-gpfs-archive/leiheao/det/ltr/dataset/depthtrack.py", line 185, in frame_list = [self._get_frame(depth_path, f_id) for ii, f_id in enumerate(frame_ids)] File "/root/siton-gpfs-archive/leiheao/det/ltr/dataset/depthtrack.py", line 158, in _get_frame img = get_rgbd_frame(color_path, depth_path, dtype=self.dtype, depth_clip=True) File "/root/siton-gpfs-archive/leiheao/det/ltr/dataset/depth_utils.py", line 15, in get_rgbd_frame rgb = cv2.cvtColor(rgb, cv2.COLOR_BGR2RGB) cv2.error: OpenCV(4.8.1) /io/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'

• dataset/DepthTrack/train/toy07_indoor_320/color/00001385.jpg is not exits! thank you if you could help me

You can check the datasets, and you'll find that something lost indeed. I deleted some incomplete categories and it works, but it definitely have an effect on metrics.