Problem to reproduce OpenLanev2 result

[hagianga21]

Hi, I am trying to reproduce the reported results on OpenLanev2. I didn't change anything. I downloaded the checkpoint and run the test script with this cmd: ./tools/test.sh plugin/mmdet3d/configs/internimage-s.py internimage-s-openlane.pth 8 --eval * --eval-options dump=True dump_dir= work_dirs

But the performance is not that good. Here is the results I got: {'DET_l': 0.0005530441, 'DET_t': 0.002188278, 'TOP_ll': 0.0, 'TOP_lt': 6.238920963764743e-05, 'OpenLane-V2 Score': 0.0026600015060585045, 'F-Score for 3D Lane': 0.0033523358535203166}

Could you please help me take a look? Thanks

[cyty98]

Got a similar result. Did you figure out what's the problem? {'DET l':0.031355385, 'DET t':0.0020301112, 'TOP ll': 5.200495277053748e-05, 'TOP lt': 0.0009464013969901333, 'OpenLane-V2 Score': 0.017840144855253124, 'F-Score for 3D Lane': 0.13003625322048777}

[hagianga21]

No, I didn't. Have you tried training it instead of using the checkpoint? Maybe something wrong in the checkpoint

[cyty98]

Not yet, that is going to take some time. Btw I noticed that the config file they provided is not completely coherent with the checkpoint file. In autonomous_driving/openlane-v2/plugin/mmdet3d/configs/internimage-s.py line23, the number of 3D control points for each lane curve is set to 5, yet the checkpoint file contains parameters for a detector head which produces 11 points. (cf figure below) I got my result by correcting this mistake in config, but as you see, it's still far from their result. Maybe there are other mistakes which lead to this poor performance?