How to training two datasets with different camera intrinsic together

[junjunxia]

Hi,

Thank you very much for release source code!

I have three basic questions

1. Can I train two datasets (with different camera intrinsic) in one mini-batch by just pass different intrinsic param to loss function?
2. In the paper, you mentioned the image could be resized during training, in which situation we need to do the image resize except the pre-train?
3. Is the V2-99 backbone model able to use in the real-time(more than 25 fps) scenario?

Thanks

[dennis-park-TRI]

Hello @junjunxia, thanks for your interest.

1. Yes, you should be able to train with two datasets with different resolution by passing intrinsics params for each dataset item. Of course there may be some amount of degrading due to the domain gap, but the DD3D is fairly robust against resoultion difference.
2. Resizing image is part of the data augmentation which is generally known to help train better model, espeically if you're training data is small.
3. Unfortunately not. V2-99 is a pretty heavy backbone, and it won't probably run in realtime.