Incorrect surface interaction results on Mac

[arpit15]

I created 3 planes at heights 2.5(top plane), 2.75(bottom plane) and 3(middle plane) from the pinhole in blender and used the depth_integrator example in the repository. The depth integrator seems to give 1.5(middle plane), 1.26(top) and 1.38(bottom plane).

I expect the depth values to be close(as distance from pinhole is slightly more than depth in world coordinate) to what I modeled in blender. May be my interpretation is wrong. Kindly let me how to get the correct depth values.

Steps to reproduce

• Download the model file
• Change the location of file in depth integrator example
• run the example

The model xml file can be found here.

[Speierers]

Hi there, Could you check that the ray.d vector is normalized in the sample_ray() method of the sensor you are using?

[wjakob]

Is the issue here perhaps just that the depth integrator will give straight-line distance norm(connecting segment) instead of the 'z' coordinate, which @arpit15 may be expecting?

[arpit15]

I am using perspective sensor for which sample_ray() method normalizes the ray.d and then multiplies with a rotation trafo. Therefore the ray.d should be a normalized vector. I expect the straight line distance of the plane center to be slightly greater for 2.55(top plane), 2.795(bottom plane) and 3(middle plane). However, current values are nowhere close. Maybe I am misinterpreting the origin. Currently, I expect the origin to be the blender cross-hair which is at (0,0,0) according the model file and blender

[dnakath]

Hi @arpit15 , are you sure that you interpret the resulting .exr in the right manner? Maybe your matplotlib normalizes your outcomes in a strange fashion? I used the depth integrator before and always received reasonable results in terms of @wjakob's norm(connection segment).

I downloaded your files and computed a depth.exr with /mitsuba2/docs/examples/02_depth_integrator/

If you look at the number to the lower right you see your ~2.75, ~3, ~2.5:

I think it might be your visualization, not Mitsuba.

[arpit15]

@dnageo thanks for your reply. I checked the numerical value in the middle of the planes by appending following code to the depth_integrator script.

image_np = np.array(bmp)
print(f'top plane:{image_np[150,900]}, middle plane:{image_np[500,900]}, bottom plane:{image_np[900,900]}')

I get following output

top plane:[1.2612995], middle plane:[1.4978386], bottom plane:[1.3853532]

Can you kindly share your code which you used to create a gif? I might be making a very obvious mistake in interpreting the results.

[dnakath]

Hi @arpit15, I have no special code to share, I just used /mitsuba2/docs/examples/02_depth_integrator/depth_integrator.py and had a look at the resultant depth.exr. You are also using this example, right?

Additionally, when I add your code, my numbers match:

# We then create a Bitmap from these values and save it out as EXR file
bmp = Bitmap(xyzaw_np, Bitmap.PixelFormat.XYZAW)
bmp = bmp.convert(Bitmap.PixelFormat.Y, Struct.Type.Float32, srgb_gamma=False)
bmp.write('depth.exr')
image_np = np.array(bmp)
print(f'top plane:{image_np[150,900]}, middle plane:{image_np[500,900]}, bottom plane:{image_np[900,900]}')

top plane:[2.5271454], middle plane:[3.0010736], bottom plane:[2.7756925]

Initially I thought you might have set the srgb_gamma=True. But when I do that, I still don't get your numbers:

top plane:[1.4973516], middle plane:[1.6125258], bottom plane:[1.5591941]

I used the packet_rgb-Version with Embree. Have you tried the AOV-integrator yet?

[dnakath]

Hi @arpit15 ,

did something new come up, or is your issue resolved and my considerations matched? In the latter case you might want to close this issue?

Cheers David

[arpit15]

Hi @dnageo thanks for checking back. I missed your last update. I checked on multiple machines. It seems that there is some discrepancy in results on mac. I tried with Embree and mitsuba2 commit d6dfb75b on ubuntu 18.04 and Mac(OS 10.15.5) Results on ubuntu top plane:[2.5271392], middle plane:[3.0010724], bottom plane:[2.7756903] Results on mac(no GPU) with and without embree top plane:[1.2612995], middle plane:[1.4978386], bottom plane:[1.3853532] Let me know if you have ideas on how to trace the bug on mac. I tried packet_rgb variant

[dnakath]

Hi @arpit15,

using the latest mitsuba2 bf4343c5

on Ubuntu 18.04 with Python 3.7.8 (default, Jun 29 2020, 05:44:46)

• packet_rgb top plane:[2.5271404], middle plane:[3.0010731], bottom plane:[2.7756927]

• packet_rgb + embree top plane:[2.5271404], middle plane:[3.0010731], bottom plane:[2.7756927]

• gpu_rgb top plane:[2.5271418], middle plane:[3.0010767], bottom plane:[2.7756891]

and on Ubuntu 20.04 with Python 3.8.2 (default, Apr 27 2020, 15:53:34)

• packet_rgb top plane:[2.5271444], middle plane:[3.0010755], bottom plane:[2.775693]

• packet_rgb + embree top plane:[2.5271444], middle plane:[3.0010755], bottom plane:[2.775693]

I still cannot reproduce your issue. So I am also quite confident now that it works on Ubuntu. So it might be a Mac-related problem. As I do not own a Mac I cannot be of further help here -- sorry!

I added my complete test folder setup depth_test.zip, just to be sure.

[greg-cohoon]

@dnakath @arpit15 : just to connect the dots, this seems related to to the comment i made on issue #14 and definitely seems like a mac issue.