Could you provide the results(epe,3 pix err) of MPI sintel based on raftstereo-sceneflow.pth?

[guoweiyu]

We prepare to citeyour paper in our new work. And we have tested the model(raftstereo-sceneflow.pth) on the MPI sintel. But the results are much worse than other datasets. We are worried about that something went wrong with our validation experiments, could you provide your experimental results?

Thanks!

[lahavlipson]

We did not evaluate our method on a held-out partition of the MPI Sintel Stereo dataset. What is your training set up (e.g. hyper parameters, training/validation set)?

[guoweiyu]

Thank you for your reply!

• We just download the raftstereo-sceneflow.pth from your google drive without any finetune and evaluate it on MPI sintel(zero-shot).
• We add the following code to evaluate_stereo.py.
• For the dataset, we use final_left and final_right as the test. We will appreciate if you could provide the result on this set up.

Thanks again!

def validate_sintel(model, iters=32, mixed_prec=False):
  
    model.eval()
    aug_params = {}
    val_dataset = datasets.SintelStereo(aug_params,sparse=True)
    torch.backends.cudnn.benchmark = True
    out_list, epe_list, elapsed_list = [], [], []
    for val_id in range(len(val_dataset)):
        _, image1, image2, flow_gt, disp,valid_gt = val_dataset[val_id]
        image1 = image1[None]
        image2 = image2[None]

        padder = InputPadder(image1.shape, divis_by=32)
        image1, image2 = padder.pad(image1, image2)

        with autocast(enabled=mixed_prec):
            _, flow_pr = model(image1, image2, iters=iters, test_mode=True)
        flow_pr = padder.unpad(flow_pr).cpu().squeeze(0)

        assert flow_pr.shape == flow_gt.shape, (flow_pr.shape, flow_gt.shape)
        epe = torch.sum((flow_pr - flow_gt)**2, dim=0).sqrt()

        epe_flattened = epe.flatten()
        val = valid_gt.flatten() >= 0.5  
        out = (epe_flattened > 3.0)
        image_out = out[val].float().mean().item()
        image_epe = epe_flattened[val].mean().item()
        logging.info(f"sintel Iter {val_id+1} out of {len(val_dataset)}. EPE {round(image_epe,4)} D1 {round(image_out,4)}")
        epe_list.append(image_epe)
        out_list.append(image_out)

    epe_list = np.array(epe_list)
    out_list = np.array(out_list)

    epe = np.mean(epe_list)
    d3 = 100 * np.mean(out_list)
    print(epe)
    print(d3)
    return {f'sintel{split}-epe': epe, f'sintel{split}-d1': d3}

[lahavlipson]

I get EPE: 17.3603 % of EPE > 3px: 13.0238 using 32 disparity updates

I'm not sure what is the difference between flow_gt and disp in your code.

[guoweiyu]

Thank you for your reply! And sorry for my late.

We get the same result. Please forget the disp in my code, it is a mistake, we have deleted it now.

Thanks again!