How to convert evaluation result to kitti label

[linchunmian]

Hi, everyone. I confuse how to convert evaluation result to kitti label when I want to predict the model on kitti test data. In the latest version of train.py, evaluate function has cancelled --pickle_result parameter, which unable to convert kitti label result automatically. Though the author suggests to use kitti_anno_to_label_file and convert_detection_to_kitti_annos in second.data.kitti_dataset, I encountered AssertionError as follow, without any instruction.

The command I use is ' python second/data/kitti_dataset.py convert_detection_to_kitti_annos --detections=results/eval_results/step_23200/result.pkl '

So, could anyone please help me to solve this problem, or any advice of how to use label convert process? Thanks in advance!

[andraspalffy]

I have the same error. Did you solve it since then?

[andraspalffy]

Maybe this will help someone, I solved it like this. In train.py, in the evaluate function, at the end (after the for loop iterating over the examples), I added these two lines:

annos = eval_dataset.dataset.convert_detection_to_kitti_annos(detections) eval_dataset.dataset.kitti_anno_to_label_file(annos, "results/labels")

which seems to convert the detectons into annotation format, and then write that into results/labels folder. Hope this helps!

[duaaany]

Maybe this will help someone, I solved it like this. In train.py, in the evaluate function, at the end (after the for loop iterating over the examples), I added these two lines:

annos = eval_dataset.dataset.convert_detection_to_kitti_annos(detections) eval_dataset.dataset.kitti_anno_to_label_file(annos, "results/labels")

which seems to convert the detectons into annotation format, and then write that into results/labels folder. Hope this helps!

I add these two lines at the end of evaluation function in train.py and get the labels results, but all the labels txt for different pointcloud imgs are the same, have you met this problem?

[andraspalffy]

Sounds to me like you have a bug in your for loop, e.g. you do not update your detections, or update with the same result. Here is my for loop with the mentioned to lines at the end.

    for example in iter(eval_dataloader):
        if measure_time:
            prep_times.append(time.time() - t2)
            torch.cuda.synchronize()
            t1 = time.time()
        example = example_convert_to_torch(example, float_dtype)
        if measure_time:
            torch.cuda.synchronize()
            prep_example_times.append(time.time() - t1)
        with torch.no_grad():
            detections += net(example)
        bar.print_bar()
        if measure_time:
            t2 = time.time()

    sec_per_example = len(eval_dataset) / (time.time() - t)
    print(f'generate label finished({sec_per_example:.2f}/s). start eval:')
    if measure_time:
        print(
            f"avg example to torch time: {np.mean(prep_example_times) * 1000:.3f} ms"
        )
        print(f"avg prep time: {np.mean(prep_times) * 1000:.3f} ms")
    for name, val in net.get_avg_time_dict().items():
        print(f"avg {name} time = {val * 1000:.3f} ms")
    with open(result_path_step / "result.pkl", 'wb') as f:
        pickle.dump(detections, f)
    result_dict = eval_dataset.dataset.evaluation(detections,
                                                  str(result_path_step))

    annos = eval_dataset.dataset.convert_detection_to_kitti_annos(detections)
    eval_dataset.dataset.kitti_anno_to_label_file(annos, "results/labels")

[duaaany]

results/labels

Thank you for your prompt reply. I put these two lines in the same place as you show, but it outputs "AttributeError: 'KittiDataset' object has no attribute 'kitti_anno_to_label_file". So I copy function kitti_anno_to_label_file in kitti_dataset.py to train.py. And change the added line to

annos = eval_dataset.dataset.convert_detection_to_kitti_annos(detections) kitti_anno_to_label_file(annos, "results/labels")

in your same place. It works but I also the same labels txt for different pointcloud imgs.

My command is python ./pytorch/train.py evaluate --config_path=./configs/all.fhd.config --model_dir=./model_dir/source_evalmodel1 --measure_time=True --batch_size=1

My all.fhd.config is eval_input_reader: { dataset: { dataset_class_name: "KittiDataset" kitti_info_path: "/home/ubuntu/new/second.pytorch/second/KITTI_DATASET_ROOT/kitti_infos_test.pkl" kitti_root_path: "/home/ubuntu/new/second.pytorch/second/KITTI_DATASET_ROOT" }

I want to get the detection result as kitti label format for the testing data in kitti dataset. Now I don't know where the problem is. Can you share your command and config, please? Thanks in advance!

[andraspalffy]

Sorry I cannot really help with this now. If the labels are the same for each pc, your problem is somewhere else, not the added two lines, at least in my opinion. I suggest checking these lines:

with torch.no_grad():   
detections += net(example)

And print what you net outputs. My assumption is that for some reason you already have the same detections here, and the problem has nothing to do with the added lines. Maybe you "example" is not loaded properly, and you always check the same pointcloud.