Training is stuck at the beginning on custom dataset

[rvrsprdx]

Hi and thanks for your work! I'm trying to train DEFT on a dataset which I have used to train CenterTrack successfully before. When running train.py the output seems promising: Creating model... Setting up train data... ==> initializing train data from ../data/s/tracking_train.json, images from ../data/data_tracking_image/training/ ... loading annotations into memory... Done (t=0.18s) creating index... index created! Creating video index! Loaded Custom dataset 202 samples Starting training... yesyes tracking/nameOfMyDataset Here the process gets stuck and nothing happens. I can see that about 800mb has been occupied by my gpu.

Do you have any idea what might be causing this issue? Thanks!

[anidh]

Hi @rvrsprdx were you able to solve this issue?

[rvrsprdx]

Yes, Iirc I had to change the import order of pytorch and opencv. Surprisingly, that fixed the issue.

[anidh]

Thanks a lot @rvrsprdx for the quick response. Were you able to detect anything with the custom dataset training? When I'm using --debug option it's Time throwing me error.

[Shelton-Zhou]

Same question. Could you please tell me where the modifications should be made? I have changed the import order of pytorch and opencv in the test.py and other files, but it didn't work.