UniDepth icon indicating copy to clipboard operation
UniDepth copied to clipboard

Unable to reproduce evaluation results from the paper

Open Nekroz05 opened this issue 5 months ago • 0 comments

I'm trying to reproduce the evaluation results presented in the paper but I'm encountering difficulty. I'd appreciate guidance on how to accurately replicate the results.

        model = UniDepthV1.from_pretrained("lpiccinelli/unidepth-v1-vitl14")

        # Move to CUDA, if any
        device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
        self.model = model.to(device)

I'm evaluating on the KITTI eigen Split and NYUv2 Official Split I'm using both v1 and v2 with vitl14 i'm using intrinsic matrix for inference

Screenshot from 2024-09-09 14-48-49

Question

  1. What specific configuration was used to achieve the results reported in the paper?
  2. Are there any additional parameters or settings I should be aware of?
  3. Is there a specific evaluation script I should be using?
  4. Are there any post-processing steps applied to the depth predictions?

My Current Result although not way off, it isn't exactly the same either, especially on RMSE

NYUv2 image

KITTI image

Nekroz05 avatar Sep 09 '24 07:09 Nekroz05