Multi view camera rendering settings

[LogWell]

How to set such a scene: https://wuminye.github.io/NHR/datasets.html

This is a relatively common schematic diagram in the current multi-view research field (such as NeRF), and I hope to add a class of such an example.

[LogWell]

The primitives can be corresponding images, or cameras like: https://www-users.cse.umn.edu/~hspark/CSci5980/csci5980_3dvision.html

[marcomusy]

if you have the individual images and orientations in space you can create

pict1 = Picture("pic1.jpg")
pict1.orientation(pic1_vector)
pict1.pos(pic1_position)
...

[LogWell]

In addition, can I define a structure that contains an image and a wireframe model of the camera, and then treat this as a whole and place it somewhere and set the orientation?

[marcomusy]

You can use an Assembly object to treat multiple objects as a single one:

from vedo import *

pic = Picture(dataurl + "images/dog.jpg")
dz = 800
x0, x1, y0, y1, _, _ = pic.bounds()
cam = [(x0 + x1) / 2, (y0 + y1) / 2, dz]

lines = Lines(
    [
        [cam, (x0, y0, 0)],
        [cam, (x1, y0, 0)],
        [cam, (x0, y1, 0)],
        [cam, (x1, y1, 0)],
    ],
    c="black",
    lw=2,
)

assem = Assembly([pic, lines, Point(cam)])
assem.orientation([0, -1, 0])

show(assem, axes=1)

[LogWell]

How to access the four corners of Picture, I do not find in picture.py and base.py, because the pic may not be axis aligned after some transformations.

[marcomusy]

You can do it with print(assem.unpack(0).bounds())

[LogWell]

Emm, Bounds and Corner are different if I do some transformations like:

pic.origin((1, 2, 2))
pic.orientation((0.2, -0.1, 0.3))

so I wonder if there are any ready-made functions that can be used.

[marcomusy]

You should transform the assem obj not the pic. I suggest not to use orientation() as it's ignoring origin() (bug). and also there is another small bug about obj.transform to being filled for pure rotations...

from vedo import *

settings.use_depth_peeling = True

pic = Picture(dataurl + "images/dog.jpg")
dz = 800
x0, x1, y0, y1, _, _ = pic.bounds()
cam = [(x0 + x1) / 2, (y0 + y1) / 2, dz]

lines = Lines(
    [
        [cam, (x0, y0, 0)],
        [cam, (x1, y0, 0)],
        [cam, (x0, y1, 0)],
        [cam, (x1, y1, 0)],
    ],
    c="black",
    lw=2,
)

corners = Points([(x0, y0), (x1,y0), (x0,y1), (x1,y1)])

assem = Assembly([corners, pic, lines, Point(cam)])
#assem.orientation([1, -1, -2]) # do not use

assem.origin(cam)
assem.rotate_y(90)
assem.rotate_z(-70)
assem.rotate_y(-90)

# T = assem.transform  # not filled (bug), workaround:
T = assem.GetMatrix()

corner_pts = corners.clone().apply_transform(T).points()
print(corner_pts)

show(assem, axes=1)

[LogWell]

Got it ^_^

[marcomusy]

...also it's probably easier if you use around to define a pivoting point instead of origin:

assem.rotate_y(+90, around=cam)
assem.rotate_z(-90, around=cam)
assem.rotate_y(-90, around=cam)

# instead of:
# assem.origin(cam)
# assem.rotate_y(+90)
# assem.rotate_z(-90)
# assem.rotate_y(-90)

to make the cam the pivoting point. I must see if there is a more intuitive/natural way for the API as these rotations are a bit tricky...

[LogWell]

How to handle this boundary when I turn to the back?

[marcomusy]

sorry I don't understand the question.

[LogWell]

When I scale this down to around 1, the image plane and the line will penetration: Note the four corners:

from vedo import *

settings.use_depth_peeling = True

pic = Picture(dataurl + "images/dog.jpg")
dz = -1000
if True:
    dz = -0.05
    dim = pic.dimensions()
    pic = pic.scale(2 / np.max(dim) * 0.1) 
x0, x1, y0, y1, _, _ = pic.bounds()
cam = [(x0 + x1) / 2, (y0 + y1) / 2, dz]

lines = Lines(
    [
        [cam, (x0, y0, 0)],
        [cam, (x1, y0, 0)],
        [cam, (x0, y1, 0)],
        [cam, (x1, y1, 0)],
    ],
    c="black",
    lw=1.2,
)

assem = Assembly([pic, lines])
assem.origin(cam)
show(assem, axes=0)

[marcomusy]

I think this is a problem with vtk that has been solved in vtk9.2.2, so I would try: pip install vtk -U

[LogWell]

After the upgrade, I still have this problem. But I can use a larger scale to temporarily circumvent this problem.