How to train a street scene and put it into the 3d software?

[cocoshe]

I just read the nerf_dataset_tips and find it in the last section:

"The colmap2nerf.py script assumes that the training images are all pointing approximately at a shared point of interest, which it places at the origin. This point is found by taking a weighted average of the closest points of approach between the rays through the central pixel of all pairs of training images. In practice, this means that the script works best when the training images have been captured pointing inwards towards the object of interest, although they do not need to complete a full 360 view of it."

It means we need to prepare the dataset like this:

But actually, when we take photos of the street(for example), I don't know how to achieve such kind of great job like this: (from ngp website) https://nvlabs.github.io/instant-ngp/assets/nrc_new_vs_old.mp4

I think in the street scene, there is no such a "object" for the center position, so I don't know how to do it.

[qhdqhd]

same question

[Yanall-Boutros]

i'm having my own unrelated issues, but if i were to take a guess, maybe make the origin focused on a point of infinity

[cocoshe]

i'm having my own unrelated issues, but if i were to take a guess, maybe make the origin focused on a point of infinity

Nice idea! I read (not that carefully) the Block-NeRF, and it seems to train for many points, and finetune for the whole scene. Maybe it's a feasible idea, I'am wondering if they have tried the "a point of infinity" theory.

[Yanall-Boutros]

i need to get my server running still, if that doesn't work, maybe cloning an image and reflecting the same image to the opposite hemisphere. Might enforce some weird sypherical symmetry on an scene that's more like a line however. Maybe moving really far away but using a strong telephoto lens with low fov could condense/collapse your far away environment to fit in frame. But then of course, you'll have a longer circumference to travel when sampling these photos from more perspectives

[meriemjabri]

i'm having my own unrelated issues, but if i were to take a guess, maybe make the origin focused on a point of infinity

Nice idea! I read (not that carefully) the Block-NeRF, and it seems to train for many points, and finetune for the whole scene. Maybe it's a feasible idea, I'am wondering if they have tried the "a point of infinity" theory.

Can you tell us what method you used?