Training with multi-view synced data

[Kartik-Teotia]

Hi,

Thanks for making the code public. I want to know if training on multi-view dynamic data would cause any obvious issue, and would that be as straight forward as having the same time encoding for the multi-view frames at each time instance?

[JaminFong]

@Kartik-Teotia Hi, thanks for your attention. I think normally training on multi-view dynamic data you mentioned will not cause any negative effects, which may make the learning easier to model multi-view consistency. However, to achieve this, multiple cameras may be needed in real scenarios. In our paper, all experiments are performed in the monocular setting, which will be more challenging to learn multi-view consistency.

P.S. Though the real-scene data of HyperNeRF is captured with two rigidly attached cameras, we split the training data to avoid same-time multi-view frames for monocular settings.