MeMOTR
MeMOTR copied to clipboard
MCG-NJU
MEMOTR evaluation on MOT17
I am trying to run the evaluation code on MOT17 using my own checkpoint for the MEMOTR model,however every time I try running the evaluation code using the below arguments,the configuration file that is loaded ends up pointing to the Dancetrack dataset. Could you tell me how to change the arguments or config file so that the evaluation can be done on MOT17.
I saw there is a specific command in eval_engine that runs the evaluation on MOT17,and worst case scenario I am thinking of directly running that command.
This is the command for evaluation
!python main.py --mode eval --data-root /content/MeMOTR/DATADIR/ --eval-mode specific --eval-model /content/MeMOTR/outputs/memotr_mot17/checkpoint_59.pth --eval-dir /content/MeMOTR/outputs/memotr_mot17/train --eval-threads 0
and this is the error log i get
2024-06-07 05:18:43.709544: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-06-07 05:18:43.763543: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-06-07 05:18:43.763595: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-06-07 05:18:43.765557: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-06-07 05:18:43.774258: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-06-07 05:18:44.912419: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
===> Running checkpoint '/content/MeMOTR/outputs/memotr_mot17/checkpoint_59.pth'
2024-06-07 05:18:49.724680: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-06-07 05:18:49.724733: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-06-07 05:18:49.725913: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-06-07 05:18:50.837143: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
Configs: {'GIT_VERSION': None, 'MODE': 'submit', 'CONFIG_PATH': './configs/train_dancetrack.yaml', 'VISUALIZE': False, 'AVAILABLE_GPUS': '0,1,2,3,4,5,6,7', 'DEVICE': 'cuda', 'OUTPUTS_DIR': './outputs/pycharm/', 'USE_DISTRIBUTED': False, 'USE_CHECKPOINT': False, 'CHECKPOINT_LEVEL': 2, 'RESUME': None, 'RESUME_SCHEDULER': True, 'MULTI_CHECKPOINT': False, 'SUBMIT_DIR': '/content/MeMOTR/outputs/memotr_mot17/train', 'SUBMIT_MODEL': '/content/MeMOTR/outputs/memotr_mot17/checkpoint_59.pth', 'SUBMIT_DATA_SPLIT': 'val', 'DET_SCORE_THRESH': 0.5, 'TRACK_SCORE_THRESH': 0.5, 'RESULT_SCORE_THRESH': 0.5, 'MISS_TOLERANCE': 30, 'USE_MOTION': False, 'MOTION_MIN_LENGTH': 3, 'MOTION_MAX_LENGTH': 5, 'MOTION_LAMBDA': 0.5, 'EVAL_DIR': None, 'EVAL_MODE': 'specific', 'EVAL_MODEL': None, 'EVAL_PORT': None, 'EVAL_THREADS': 1, 'EVAL_DATA_SPLIT': 'val', 'DATASET': 'DanceTrack', 'USE_MOTSYNTH': None, 'USE_CROWDHUMAN': None, 'MOTSYNTH_RATE': None, 'DATA_ROOT': '/content/MeMOTR/DATADIR/', 'DATA_PATH': None, 'NUM_WORKERS': 4, 'BATCH_SIZE': 1, 'ACCUMULATION_STEPS': 1, 'COCO_SIZE': False, 'OVERFLOW_BBOX': False, 'REVERSE_CLIP': 0.0, 'BACKBONE': 'resnet50', 'HIDDEN_DIM': 256, 'FFN_DIM': 2048, 'NUM_FEATURE_LEVELS': 4, 'NUM_HEADS': 8, 'NUM_ENC_POINTS': 4, 'NUM_DEC_POINTS': 4, 'NUM_ENC_LAYERS': 6, 'NUM_DEC_LAYERS': 6, 'MERGE_DET_TRACK_LAYER': 1, 'ACTIVATION': 'ReLU', 'RETURN_INTER_DEC': True, 'EXTRA_TRACK_ATTN': False, 'AUX_LOSS': True, 'USE_DAB': True, 'UPDATE_THRESH': 0.5, 'LONG_MEMORY_LAMBDA': 0.01, 'PRETRAINED_MODEL': 'dab_deformable_detr.pth', 'SAMPLE_STEPS': [6, 10, 14], 'SAMPLE_LENGTHS': [2, 3, 4, 5], 'SAMPLE_MODES': ['random_interval'], 'SAMPLE_INTERVALS': [10], 'SEED': 42, 'EPOCHS': 20, 'ONLY_TRAIN_QUERY_UPDATER_AFTER': 20, 'DROPOUT': 0.0, 'NUM_DET_QUERIES': 300, 'TP_DROP_RATE': 0.0, 'FP_INSERT_RATE': 0.0, 'LR': 0.0002, 'LR_BACKBONE': 2e-05, 'LR_POINTS': 1e-05, 'WEIGHT_DECAY': 0.0005, 'CLIP_MAX_NORM': 0.1, 'LR_SCHEDULER': 'MultiStep', 'LR_DROP_RATE': 0.1, 'LR_DROP_MILESTONES': [12], 'MATCH_COST_CLASS': 2, 'MATCH_COST_BBOX': 5, 'MATCH_COST_GIOU': 2, 'LOSS_WEIGHT_FOCAL': 2, 'LOSS_WEIGHT_L1': 5, 'LOSS_WEIGHT_GIOU': 2, 'AUX_LOSS_WEIGHT': [1.0, 1.0, 1.0, 1.0, 1.0]}
Traceback (most recent call last):
File "/content/MeMOTR/main.py", line 124, in <module>
main(config=merged_config)
File "/content/MeMOTR/main.py", line 109, in main
submit(config=config)
File "/content/MeMOTR/submit_engine.py", line 195, in submit
train_config = yaml_to_dict(path=path.join(config["SUBMIT_DIR"], "train/config.yaml"))
File "/content/MeMOTR/utils/utils.py", line 53, in yaml_to_dict
with open(path) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/content/MeMOTR/outputs/memotr_mot17/train/train/config.yaml'
mv: cannot stat '/content/MeMOTR/outputs/memotr_mot17/train/val/tracker': No such file or directory
/content/MeMOTR/TrackEval/trackeval/eval.py:99: SyntaxWarning: "is not" with a literal. Did you mean "!="?
seq_key is not 'COMBINED_SEQ'}
/content/MeMOTR/TrackEval/trackeval/utils.py:137: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if (len(current_values) == len(keys)) and seq is not '':
/content/MeMOTR/TrackEval/trackeval/datasets/mot_challenge_2d_box.py:151: SyntaxWarning: "is" with a literal. Did you mean "=="?
if self.config["SEQMAP_FOLDER"] is 'None':
Eval Config:
USE_PARALLEL : True
NUM_PARALLEL_CORES : 8
BREAK_ON_ERROR : True
RETURN_ON_ERROR : False
LOG_ON_ERROR : /content/MeMOTR/TrackEval/error_log.txt
PRINT_RESULTS : True
PRINT_ONLY_COMBINED : False
PRINT_CONFIG : True
TIME_PROGRESS : True
DISPLAY_LESS_PROGRESS : False
OUTPUT_SUMMARY : True
OUTPUT_EMPTY_CLASSES : True
OUTPUT_DETAILED : True
PLOT_CURVES : False
MotChallenge2DBox Config:
PRINT_CONFIG : True
GT_FOLDER : /content/MeMOTR/DATADIR/DanceTrack/val
TRACKERS_FOLDER : /content/MeMOTR/outputs/memotr_mot17/checkpoint_59_tracker
OUTPUT_FOLDER : None
TRACKERS_TO_EVAL : ['']
CLASSES_TO_EVAL : ['pedestrian']
BENCHMARK : MOT17
SPLIT_TO_EVAL : val
INPUT_AS_ZIP : False
DO_PREPROC : True
TRACKER_SUB_FOLDER :
OUTPUT_SUB_FOLDER :
TRACKER_DISPLAY_NAMES : None
SEQMAP_FOLDER : None
SEQMAP_FILE : /content/MeMOTR/DATADIR/DanceTrack/val_seqmap.txt
SEQ_INFO : None
GT_LOC_FORMAT : {gt_folder}/{seq}/gt/gt.txt
SKIP_SPLIT_FOL : True
no seqmap found: /content/MeMOTR/DATADIR/DanceTrack/val_seqmap.txt
Traceback (most recent call last):
File "/content/MeMOTR/TrackEval/scripts/run_mot_challenge.py", line 84, in <module>
dataset_list = [trackeval.datasets.MotChallenge2DBox(dataset_config)]
File "/content/MeMOTR/TrackEval/trackeval/datasets/mot_challenge_2d_box.py", line 82, in __init__
self.seq_list, self.seq_lengths = self._get_seq_info()
File "/content/MeMOTR/TrackEval/trackeval/datasets/mot_challenge_2d_box.py", line 157, in _get_seq_info
raise TrackEvalException('no seqmap found: ' + os.path.basename(seqmap_file))
trackeval.utils.TrackEvalException: no seqmap found: val_seqmap.txt
Traceback (most recent call last):
File "/content/MeMOTR/main.py", line 124, in <module>
main(config=merged_config)
File "/content/MeMOTR/main.py", line 111, in main
evaluate(config=config)
File "/content/MeMOTR/eval_engine.py", line 36, in evaluate
metrics = eval_model(model=config["EVAL_MODEL"], eval_dir=eval_dir,
File "/content/MeMOTR/eval_engine.py", line 118, in eval_model
with open(metric_path) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/content/MeMOTR/outputs/memotr_mot17/checkpoint_59_tracker/pedestrian_summary.txt'
not sure at which point the dancetrack configuraiton was referred to here,because I am using the MOT17 dataset.
For inference on MOT17, you should add a runtime option --config-path <your training config file path> to the submitting and evaluation scripts. Sorry for the insufficient details. I will make this clearer in our documents later.
Sorry to bother. If use the main.py provided for the evaluation, there will always be a lack of val on mot17, whether there are clearer steps
Sorry to bother. If use the main.py provided for the evaluation, there will always be a lack of val on mot17, whether there are clearer steps
Hi, I've replied to you at #29. For MOT17, this codebase only supports submitting for the test set rather than the evaluation.