Camera trajectory different to the trajectory in CSV file

[jprodriguezg]

Hello,

I checked that among the options of the configuration file there is the possibility to define the trajectory of the camera using a CSV file. I tried one of your examples, but the camera does not follow exactly the trajectory defined in the CSV file. In the code, the trajectory is read, but then a spline is computed as the trajectory to follow.

I am wondering if in your code there is a way in which the camera follows exactly the trajectory defined in the CSV file. Is this possible?

[supitalp]

Hello, Yes, a (continuous) spline in SE(3) is computed from the discrete poses specified in the CSV file. This is a design choice we made: it allows to compute the camera velocity (resp. acceleration) in closed-form, which are used to approximate the motion field on the sensor (which is in turn used to decide when the next frame should be synthesized), and also to simulate the inertial measurement unit accurately. To make the spline fit a bit closer to the camera poses, you may increase the number of spline segments (parameter --trajectory_num_spline_segments). Keep in mind that the higher this number, the longer it will take at startup to estimate the spline. A reasonable number to start experimenting would be around 200 or so, depending on the length of your trajectory, and the desired accuracy.

[jprodriguezg]

Hello,

I did some tests by increasing the number of spline segments as you suggested. The computed spline approximates the position correctly in most of the cases. However, the orientation of the spline does not correspond to the orientation of the CSV file. The camera rotates in yaw from left to right and vice-versa while following the position of the trajectory. It seems like a pre-programmed behavior. I tried with different trajectories and the results are always the same. The only way to keep the orientation still is defining an input trajectory with qx,qy,qz = 0 and qw = 1

Do you know about this issue? Perhaps it works in this way given your design. Or is it an issue from the spline generator?

Thanks!

[supitalp]

Hello, Could you please send me the faulty CSV file so I could have a look? It may be a problem with the spline generator indeed, which I have observed sometimes has problems around singularities. Thanks.

[jprodriguezg]

Hello,

Sure, attached I send you the CSV file with the trajectory that I am using for my test (test_yaw.csv). Additionally, I add a file (test_yaw_output.csv) with the output from the event camera simulator. I just saved the pose commands that the package sends to the UE4 binary.

I also checked the orientation produced by the spline generator using one of your sample files (quadrotor_circles.csv). The result is the same, the orientation returned by the spline generator does not correspond to the orientation given by the input file.

test_trajectory.zip

[504298]

Hello,

I'm facing a similar problem. In my case, splines don't follow some straight lines, curves or sines quite well, though I've tried with 200, 400, 800 and 1600 spline segments. It seems that splines have problems with changes in all of the variables at the same time and with trajectories other than straight lines. For example, I made this two simulations:

Simulation 1:

• Input trajectory of 3 seconds: straight line in x, sines in y, z, yaw, pitch and roll (I transformed them later to quaternions). Sines of 1 or 2 Hz
• Trajectory defined for 120 points in each variable
• 200 spline segments along the trajectory
• Splines don't fit well sines

Simulation 2:

• Same trajectory as simulation 1, but only along the first 0.1 seconds
• Trajectory defined for 5 points in each variable
• 8 splines segments along the trajectory
• Splines fit well these portions of the sines, regardless the reduction of spline segments

Anyway, thanks for uploading ESIM; it's a great work and we appreciate it!

[supitalp]

Hello, Does the fix suggested here: https://github.com/uzh-rpg/rpg_esim/issues/33#issuecomment-513663158 fix your issue?