FFHQ images are not sampled from the same camera coordinate systems?

[blandocs]

Hi authors, thank you again for the great work :) 👍

I found some issues when I tried to align geometry rendering with rgb images.

I assume that all cameras of the FFHQ images have the same coordinate system in the first place.

It means that initially all cameras look at the same direction(e.g, [0, 0, 2.7]) and we can rotate each camera according to the face angles.

But when I calculate a translation matrix using the rotation matrix of the dataset and euler angles, it differs from the original dataset you provided as shown in the bold ones.

extrinsics from the original dataset tensor([[[ 0.992, 0.014, -0.125, 0.317], [-0.004, -0.989, -0.145, 0.388], [-0.125, 0.144, -0.982, 2.653], [ 0.000, 0.000, 0.000, 1.000]]])

calculate extrinsics from rotation matrix of the dataset using euler angles tensor([[[ 0.992, 0.014, -0.125, 0.337], [-0.004, -0.989, -0.145, 0.392], [-0.125, 0.144, -0.982, 2.650], [ 0.000, 0.000, 0.000, 1.000]]])

When I estimate initial camera direction from the calculated extrinsics, I also found that the initial direction vectors are different in each image like below one.

# img 1
tensor([[[ 0.003],
         [-0.017],
         [ 2.700]]])
# img 2
tensor([[[ 0.020],
         [-0.004],
         [ 2.700]]])
# ...

Is this a bug or am I missing something?

Thank you in advance :)

[luminohope]

Hi blandocs,

we have a preliminary in the wild preprocessing script and you will see some translation adjustment etc we did in the script. Perhaps it might be related? https://github.com/NVlabs/eg3d/blob/FFHQ_preprocess/dataset_preprocessing/ffhq/preprocess_in_the_wild.py