novel view generation issue. overfitting?

[hengfei-wang]

Hi,

Thank you for releasing the code. It is quite helpful to me.

I used my dataset which contains 140 iamges with different time stamps. But the images only from 14 training camera poses. Then I found the network can't generate good novel view images. I used the same dataset on D-nerf and it worked well. Are 14 training views too sparse for training the network? Have you found the same problem?

Thanks

[JaminFong]

Thanks for your attention! I have not yet met this problem. Could you show the failed generated images? Then I can better understand the problem.

[hengfei-wang]

Yes. I interpolated camera poses from pose1(the first image) to pose2(the second image). You can see that only trained poses can have a good render performance.

[dsvilarkovic]

Have you solved this? I might have the same problem, but it makes no sense to me to not be able to generate novel views?

[hengfei-wang]

Hi @dsvilarkovic,

I haven't solved the problem. If you have any thoughts on this, please let me know.

[dsvilarkovic]

I solved it by converting my llff format camera poses to nerf/blender .json format with the script I found in torch-ngp https://github.com/ashawkey/torch-ngp/blob/main/scripts/llff2nerf.py

[hengfei-wang]

@dsvilarkovic Thank you for your solution. It solved my problem (not perfect but at least make sense). But I still don't get it. What cause this difference?

[dsvilarkovic]

To be honest, I do not know. \

[hengfei-wang]

OK, thanks.