About the intrinsic and extrinsic of the camera

[Moreland-cas]

Hi there, fantastic work—really impressive!

I have a few questions about your method:

1. Does the approach assume that the camera extrinsic remains constant throughout the process?
2. Is there a specific way we need to set or configure the camera intrinsics?
3. Are the output 3D tracked points expressed in the coordinate frame of the first image?

Looking forward to your response. Thank you for your time and for sharing this great project!

[nicolas-schreiber]

I'd also like to know that, especially since I do have that information and at the moment all my objects do look slightly flattened, which I assume is connected to the intrinsics

[KaKituken]

Since the model is trained on ZeoDepth, I guess the intrinsic is aligned with ZeoDepth. I found this in ZeoDepth's repo

# geometry.py from ZeoDepth
def get_intrinsics(H,W):
    """
    Intrinsics for a pinhole camera model.
    Assume fov of 55 degrees and central principal point.
    """
    f = 0.5 * W / np.tan(0.5 * 55 * np.pi / 180.0)
    cx = 0.5 * W
    cy = 0.5 * H
    return np.array([[f, 0, cx],
                     [0, f, cy],
                     [0, 0, 1]])

But not sure whether it matches the final tracking 3D result, I'm verifying it.

[OliEfr]

@henry123-boy Any input from your side? :)

[Yefancy]

They assume the intrinsic according to the video resolution here https://github.com/henry123-boy/SpaTracker/blob/main/demo.py#L210-L220 fov ≈ 50~55

[henry123-boy]

Thanks for following our project. The extrinsic and intrinsic did not be considered in SpatialTrackerV1. But the good news is that our SpatialTrackerV3 is going to release in a week Spatrack3. V3 will be a unified model to produce video depth, camera intrinsic & extrinsic and 2D/3D tracks at once. Hope this will help you guys!

[Yefancy]

Thanks for following our project. The extrinsic and intrinsic did not be considered in SpatialTrackerV1. But the good news is that our SpatialTrackerV3 is going to release in a week Spatrack3. V3 will be a unified model to produce video depth, camera intrinsic & extrinsic and 2D/3D tracks at once. Hope this will help you guys!

good to know. btw, would you provide the training code this time?