[request] JPG DepthMaps

[sfjuocekr]

MeshRoom currently does not handle JPG's that contain a depthmap, like the images you get from phones with a depth sensor like the HMD Nokia 9 PureView.

On Linux the only reliable way I found to extract the depth image was to use:

exiftool -DepthImage image -b > image.depth convert -depth 8 -size 4032x3016 image.depth depth_image.jpg

These images -should- be helpful to MeshRoom? But I can't seem to get it to work, this does work with Metashape as far as I can tell. I photographed a bunch of stones next to my house, MeshRoom can't make anything out of it but with Metashape I can get a fairly decent model + texture from just using the images and from the depth maps.

I can upload the images somewhere if anyone wants to experiment!

[simogasp]

yes please if you have a small dataset you can upload it here, maybe someone will be interested in working on it. But for the moment the depth map is not used, it could be a very interesting feature to add. (it also depends on the quality of the depth maps)

[sfjuocekr]

http://downloads.sjoer.org/photos.tar.bz2

Some of the depthmaps are botched, when I set it to sequential it finds the most cameras in Meshroom and with Metashape it usually fails to align just one (the one that is shot near horizontally with the car in the background).

#!/bin/sh

for f in $(ls -1 *.jpg)
do
        d="${f%.jpg}";
        exiftool -DepthImage $f -b > depth_$d.depth
        convert -depth 8 -size 4032x3016 depth_$d.depth depth_$d.png
done

This works to get you depth maps and convert them to PNG to easily view with GIMP and the likes.

If anyone wants to re-use my pictures, feel free to do so! I recon someone more capable could make some nice textured stones out of them! I'll upload the output OBJ + texture I made myself in a few minutes from now!

edit:

http://downloads.sjoer.org/output.tar.bz2

[natowi]

Similar to https://github.com/alicevision/meshroom/issues/188#issuecomment-575887872, https://github.com/alicevision/meshroom/issues/623 and https://groups.google.com/g/alicevision/c/VUNMVsr25ZY/m/hWTZIGXtBAAJ (+ https://github.com/alicevision/meshroom/issues/255)

[CeciliaPYY]

@sfjuocekr Hello, may I ask where can I find the tutorial about sending depth maps together with rgb images into Metashape to 3d reconstruction??? Thanks a lot!!!

[natowi]

@CeciliaPYY Sorry, there is no such tutorial. You should ask this in the Agisoft forum https://www.agisoft.com/forum/index.php?topic=9729.0 https://www.agisoft.com/forum/index.php?topic=11370.0 but it does not look like it is possible. Maybe Metashape really supports embedded depth maps, but I could not find any mentions on this.

[CeciliaPYY]

@natowi Anyway, thanks again for your reply, I'll check it out by myself in related tutorial.

[sfjuocekr]

Sorry a bit late, but yea there is no such tutorial as there is no support for it.

Metashape has no support either, they all generate depth maps after resolving scene positions and thus also "depth".

[Vineg]

Hi, sfjuocekr!

Could we use your dataset as an example in the tutorial( https://agisoft.freshdesk.com/support/solutions/articles/31000162212-smart-cameras-with-depth-sensor-data-processing )?

[sfjuocekr]

Yes