Rotation matrices are being interpreted from cursor / origin

[VisionaryMind]

I have some basic Bundler data which consists of cameras in a straight line on X (east-west) axis, then going up on Y (vertical), rotating 90 degrees on Y to the right and then moving straight forward further along the X axis. The problem I see is that when heading shifts 90 degrees to the right (on the same vertical axis), the add-on is somehow interpreting that as a translation. The cameras are shown to rotation around the cursor / world origin instead of around their respective local axes.

I attempted to fix this problem by switching Blender's transform orientation to "gimbal" and transform pivot point to "individual origins", but those do not seem to have an effect on how the rotation matrix is processed. Here is a small sample of that rotation:

Translation values are Z,Y,X which correspond to axis north-south, altitude / vertical, and east-west. The rotation should be happening on the vertical axis (middle / Y).

1614.15662 0 0 0.99383223 0.00806891 -0.110600464 -0.0017907226 0.99838704 0.056746747 0.11087996 -0.056198686 0.99224365 -0.071071476 -0.11664623 2.469451

1614.15662 0 0 0.97913843 0.021447742 -0.20205948 -0.008532568 0.9978766 0.06457322 0.20301536 -0.061502036 0.9772423 -0.07967277 -0.12266238 2.4893844

1614.15662 0 0 0.9516065 0.027723696 -0.30606624 -0.0058047106 0.9973665 0.07229442 0.30726448 -0.06701922 0.9492613 -0.08839952 -0.13129313 2.523389

1614.15662 0 0 0.89873374 0.03140694 -0.43736857 -0.001721531 0.99767673 0.068104506 0.4384914 -0.06045488 0.8966997 -0.09061018 -0.1356472 2.5761647

1614.15662 0 0 0.82490796 0.03398874 -0.56424433 0.0020853977 0.9980009 0.063165985 0.5652633 -0.053282805 0.82318807 -0.07985141 -0.13775776 2.6344905

1614.15662 0 0 0.70870894 0.03639164 -0.7045617 0.013551228 0.99778235 0.0651679 0.7053708 -0.055732753 0.7066441 -0.055255346 -0.14196688 2.7103112

1614.15662 0 0 0.5863702 0.04462327 -0.8088132 0.018017095 0.9975161 0.068096206 0.8098429 -0.05450206 0.5841098 -0.020925561 -0.14822748 2.7860377

1614.15662 0 0 0.45274648 0.057762425 -0.88976634 0.009771731 0.997518 0.06972974 0.8915857 -0.04026445 0.4510583 0.029267171 -0.15076374 2.8668447

I am aware my matrices are not "standard". This is a horizontal orientation / right-handed custom scanner. Nonetheless, we should not be seeing any translation during camera import if the those last three coordinates are stable.

[stuarta0]

Here's a definition of the transform being applied: https://github.com/simonfuhrmann/mve/wiki/Math-Cookbook#extrinsic-parameters

i.e. t = -R * c where t is the final camera location, R is the camera rotation matrix and c is the camera location in the bundler file.

If you're comfortable modifying the installed addon, you could try swizzling the location vector here: https://github.com/stuarta0/blender-photogrammetry/blob/7b840cb9c94320c2a34a2f1625f16398f13c98dc/bundler/extract.py#L37

to

translation = Vector(map(float, lines[idx + 4].split())).yzx

I had a quick test and it looks more like you're describing: rotating near a single point/axis rather than all the cameras arcing through the scene.

If this does end up being the cause, I can add an extra field for parsing bundler files to swizzle the location on import.

[stuarta0]

See here for blender directory structure including addon location: https://docs.blender.org/manual/en/latest/advanced/blender_directory_layout.html

[VisionaryMind]

I applied the change to the translation per your recommendation, and while this does align the cameras properly with the grid, as soon as rotations are applied, those translations change. As a test, I applied a 180 degree turn on the Y axis and a 90 degree turn on the Z using this code modification (requires numpy imports for dot, asarray, and float32):

tmatvals = [[0.268154167251667,-0.893996663600558,0.358975358564673],
    [-0.535021305035885,-0.44807361612917,1.09284402786673],
    [0.80115263573383,0,-0.598460069057858]]

#  See "3 3D Coordinate axes rotation matrices" at
# https://sites.google.com/site/glennmurray/Home/rotation-matrices-and-formulas/rotation-about-an-arbitrary-axis-in-3-dimensions

transmat = asarray(tmatvals,dtype=float32)
total_cameras, total_points = map(int, lines[1].split())
for i in range(int(total_cameras)):
    # each camera uses 5 lines
    idx = 2 + i * 5
    # read data
    focal, k1, k2 = list(map(float, lines[idx].split()))
    
    rots = asarray([lines[idx + (j)].split() for j in range(1,4)],dtype=float32)
    idxs = rots.dot(transmat)

    rotation = []
    rotation.append(map(float, idxs[0]))
    rotation.append(map(float, idxs[1]))
    rotation.append(map(float, idxs[2]))
    translation = Vector(map(float, lines[idx + 4].split())).yzx 

The non-rotationally transformed camera positions are in the shape of a "+" with a long X-axis and a short vertical Y. After applying the rotation transform above, the actual positions of the cameras all start leaning in the direction of the X-axis. Any further, they would be straightened out into a single line.

As it stands, if you place a camera on a tripod and spin it around 360 degrees, that continuous heading change will turn out as a translational displacement in Blender using your add-on. Again, there should be no change of camera position just by rotating around on a single axis (as with panoramas). Rotation matrices should be interpreted locally -- per camera. This is the only way to maintain coordinate fidelity. Translation should be the only movement in 3D space. Rotation is what happens to each camera, around its own origin, in relation to the scene after its position (along 3 planes) has been established. It makes no sense that a camera's position would change based on it's rotation. This should never happen.

I'll continue to troubleshoot. Perhaps there is a way to force local origin for rotations. I think the problem is that Blender is interpreting camera rotations around the world origin, and that is what is causing camera positions to shift. We do a lot of hand-held scans that have wild variability in heading, pitch, and roll .... unlike other photogrammetric scans that stabilize one or more of those axes.

[stuarta0]

It sounds like you're assuming the translation vector in the bundler file is absolute as per traditional 3D graphics? As far as I'm aware, the final 3D location of the camera is a product of the transposed camera rotation matrix and it's translation value, as defined in the bundler spec. Further details can be found here: http://www.cs.cornell.edu/~snavely/bundler/bundler-v0.4-manual.html

Specifically section VI: Output format and scene representation

We use a pinhole camera model; the parameters we estimate for each camera are a focal length (f), two radial distortion parameters (k1 and k2), a rotation (R), and translation (t), as described in the file specification above. The formula for projecting a 3D point X into a camera (R, t, f) is: ...snip...

Finally, the equations above imply that the camera viewing direction is: R' * [0 0 -1]' (i.e., the third row of -R or third column of -R')

(where ' indicates the transpose of a matrix or vector).

and the 3D position of a camera is -R' * t .

[stuarta0]

For the record I'm just following the formulas and formats that other much smarter people came up with :)