NVlabs
How does FoundationPose pick the first frame for the pose estimate?
Hi! This is some great work @wenbowen123! It's turning out to be very useful for my research.
I was wondering how the initial frame is estimated/placed? I was able to get it to run for a novel object using the model based method. I just want my initial frame to be rotated by 45 degrees in order to have it align in a more visually intuitive way with the object I'm tracking.
For example, what if I wanted this initial triad to be rotated in such a way that the blue axis aligns with the text and the red points at the cap but the green axis is not visible?
you can change it in the mesh. For the OBJ file, you can open it in Meshlab and rotate it to the way you want, then over-write the original one.
Could you point me to where in the code the initialization is done?
there is no code needed for this. As I said, you just open the OBJ file in Meshlab and rotate it to the way you want, then over-write the original one.
Here's what my obj file looks like which is axis aligned and oriented the way I want.
Here's what the tracking looks like:
I would expect that this should have foundation pose pick an axis aligned initial pose. I'm not sure why this isn't the case - do you have any suggestions?
https://github.com/NVlabs/FoundationPose/blob/main/run_demo.py#L69 what if you change this line to
center_pose = pose
I made the change and this is what happens (images from two runs with the same initial pose):
From different initial pose:
It seems like they are axis-aligned now? It's hard to tell from your mesh editing tool how the object is oriented.
It isn't consistent. It sometimes picks the initial pose to be axis aligned but not always. If you look at the first two images, blue and green axes are wrong.
Is it expected to differ with every experiment? If yes, is there a way to ensure it doesn't?
@sharanyashastry @wenbowen123 have you been able to find a resolution to this problem? I made the change that @wenbowen123 suggested but still the problem persists in some cases. I also have been experiencing something similar where the coordinate frame is sometimes flipped.
@wenbowen123 , I am also getting the same issue as @sharanyashastry mentioned , initial pose estimate axis convention are not consistant across multiple rerun , Could you suggest any way to solve this ?
