tusimple-benchmark
tusimple-benchmark copied to clipboard
questions about tusimple evaulation logic
hi, authors, I have several questions about tusimple evaluation logic.
-
It seems that the evaluation logic doesn't consider whether the prediction and ground truth match with each other, or just default prediction and gt match with each other.
-
when using pt_thresh=0.85 to calculate num of matched, it seems that the evaluation logic only consider the max(accs) among the lines in single image. As shown in the below:
max_acc = np.max(accs) if len(accs) > 0 else 0
if max_acc < LaneEval.pt_thresh:
fn += 1
else:
matched += 1
The above are all my questions, I am looking forward to your reply.
Best, Max