ODM icon indicating copy to clipboard operation
ODM copied to clipboard

Crashing during undistortion step

Open elunty opened this issue 2 years ago • 4 comments

How did you install ODM? (Docker, installer, natively, ...)?

Windows Installer (v3.3.4)

What is the problem?

Program silently crashes during undistortion of images

What should be the expected behavior? If this is a feature request, please describe in detail the changes you think should be made to the code, citing files and lines where changes should be made, if possible.

Not crashing would be good, but giving more logs explaining why it happened would be useful

Log before it "crashes":

[INFO] Undistorting F:\Testing\opendronemap\sample #1\opensfm ... 2023-12-20 09:21:54,986 DEBUG: Undistorting image panoramic_2023-10-09_16-10-37_000368.jpg 2023-12-20 09:21:54,987 DEBUG: Undistorting image panoramic_2023-10-09_16-10-37_000386.jpg 2023-12-20 09:21:54,987 DEBUG: Undistorting image panoramic_2023-10-09_16-10-37_000378.jpg 2023-12-20 09:21:54,988 DEBUG: Undistorting image panoramic_2023-10-09_16-10-37_000369.jpg 2023-12-20 09:21:54,988 DEBUG: Undistorting image panoramic_2023-10-09_16-10-37_000387.jpg 2023-12-20 09:21:54,987 DEBUG: Undistorting image panoramic_2023-10-09_16-10-37_000391.jpg 2023-12-20 09:21:54,989 DEBUG: Undistorting image panoramic_2023-10-09_16-10-37_000379.jpg 2023-12-20 09:21:54,988 DEBUG: Undistorting image panoramic_2023-10-09_16-10-37_000390.jpg 2023-12-20 09:21:54,987 DEBUG: Undistorting image panoramic_2023-10-09_16-10-37_000392.jpg 2023-12-20 09:21:54,988 DEBUG: Undistorting image panoramic_2023-10-09_16-10-37_000389.jpg 2023-12-20 09:21:54,989 DEBUG: Undistorting image panoramic_2023-10-09_16-10-37_000393.jpg 2023-12-20 09:21:54,988 DEBUG: Undistorting image panoramic_2023-10-09_16-10-37_000367.jpg

(venv) C:\ODM> (venv) C:\ODM> `

elunty avatar Dec 20 '23 16:12 elunty

How can we reproduce this? What steps did you do to trigger the problem? If this is an issue with processing a dataset, YOU MUST include a copy of your dataset AND task output log, uploaded on Google Drive or Dropbox (otherwise we cannot reproduce this).

pierotofy avatar Dec 20 '23 16:12 pierotofy

Sorry, im not allowed to share the dataset. Is there not anywhere i can look for extra logs about why it is crashing?

elunty avatar Dec 20 '23 17:12 elunty

Example dataset here: https://drive.google.com/file/d/1Gr424P_ZaVM50iK3Ydicu2SZr9xgjLwz/view?usp=sharing

Ongoing discussion here: https://community.opendronemap.org/t/processing-crashes-while-undistorting-panoramic-images-with-no-extra-logs/18683/

In short, it appears that with 8K 360 data, undistort becomes the memory bottleneck rather than our usual texturing step.

@elunty -- I don't know if you want to attempt a patch, but just in case, I'll re-open the issue now that we have an example dataset.

Now testing on Linux with a node that is limited to 32GB to see if we can replicate across operating systems.

smathermather avatar Dec 22 '23 02:12 smathermather

I get crashes at different places each time with 32 and 64GB, but I'm not sure how your virtual memory is configured on Windows, so no extra insights from my testing.

smathermather avatar Dec 22 '23 13:12 smathermather