Thermal support for DJI M3T [WIP]

[pierotofy]

trafficstars

Started adding thermal support for the DJI M3T, but it's not clear how to interpret the raw thermal data information.

This is being investigated at https://exiftool.org/forum/index.php?topic=11401.45 as well.

[smathermather]

Related: https://community.opendronemap.org/t/celsius-values-image-looks-good-but-celsius-ranges-are-not-correct/17736/1 with possibly available processed data with corrected values.

[aviosmedia]

Per the request on the forum, here is a link to some files for testing and calibration of the Celsius values.
link

The folder contains a sub-folder with images taken using a Mavic 3T, and an image in the main folder showing the results, along with temperature ranges, as processed in another system.

Hopefully, that helps move things forward.

Thanks for all you do!

[smathermather]

Hopefully, that helps move things forward.

I downloaded these, so if they are needed after the link expires, I have a copy. @aviosmedia -- can you share the final tif dataset from the other software. The screenshot is helpful, but being able to interrogate the final raw values might be helpful.

Thanks so much for sharing quickly.

[aviosmedia]

Hopefully, that helps move things forward.

I downloaded these, so if they are needed after the link expires, I have a copy. @aviosmedia -- can you share the final tif dataset from the other software. The screenshot is helpful, but being able to interrogate the final raw values might be helpful.

Thanks so much for sharing quickly.

Hi - I added another folder to the same link with the tif files @smathermather so you can download again (or just download the new files).

Hopefully, that gives you what you need.

Thanks!

[ambarja]

Is there a solution to this problem? I remember I opened a topic in the WebODM community and also added some data as an example.

https://community.opendronemap.org/t/is-possible-calibration-the-thermal-images-of-dji-mavic-3t/16179

[Saijin-Naib]

Not yet. This is a large undertaking, unfortunately.

[henfrydls]

Hello everyone,

@pierotofy @smathermather I haven't been using ODM and WebODM for very long. However, I've been following this topic for some months now.

Would it work to convert the images to TIFFs and copy the metadata afterward? I have seen several people doing it this way and saying it works for them. Most of them use DJI SDK to convert the images from DJI Mavic 3T to TIFF.

I'm not an expert in this, but depending on what you think, I could try it out.

[pierotofy]

Progress! :clinking_glasses:

The raw thermal data is encoded in the APP3/4/5 sections, but DJI applies some sort of correction based on temperature, distance, emissivity and humidity. The uncorrected data in Celsius can be recovered with:

celsius = ((raw_int16 >> 2) * 0.0625) - 273.15

from PIL import Image

# ... load exif data

thermal_data = base64.b64decode(j["ThermalData"][len("base64:"):])
thermal_data_buf = np.frombuffer(thermal_data, dtype=np.int16)

celsius = np.right_shift(thermal_data_buf, 2).astype(np.float32)
celsius *= 0.0625
celsius -= 273.15

rows = 512
cols = 640
im = celsius.reshape((rows, cols))

img_thermal = Image.fromarray(im)
img_thermal.save('/datasets/dji_thermal/out.tiff')

I'm currently trying to understand how the correction is applied.

[aviosmedia]

If you need data from an M3T to help with please let me know

[pierotofy]

Quick update on this; DJI does a pretty obscure calibration process. Between offset 0x100 and 0x1D00 of the APP5 segment they store calibration data, which look like down-sloping curves (I counted 56 curves with 128 items in each curve), or 512 rows of data with 14 elements in each (7168 int16 numbers total). Unfortunately it's anyone's guess what these numbers do and trying to infer this information from the DJI SDK is a big undertaking.

I think the long term solution is to run DJI's own SDK on the input images prior to sending them to ODM. I started work on a friendly wrapper for it at https://github.com/uav4geo/DJI-Thermal-Tools which does that, preserving EXIF/XMP tags etc.. I don't think ODM will support built-in radiometric calibration for these sensors, unless DJI steps forward and tells us how to interpret this data. :shrug:

[henfrydls]

There are many friendly wrapper that have been built in the meawhile. I have test some myself, we even discuss some of it on a forum in OpenDroneMap. As of now, the one that had worked the best for me is https://github.com/MiroRavaProj/DJI-Tools-and-Stuff by MiroRavaProj. I could be a great idea to explore some of the work done by the comminuty, while someone comes out with a anti-DJI SDK solution.

[pierotofy]

First release is out: https://github.com/uav4geo/Thermal-Tools

Setup: https://github.com/uav4geo/Thermal-Tools/releases/download/v1.0.0/Thermal_Tools_Setup.exe

[pierotofy]

And for Linux: https://github.com/uav4geo/Thermal-Tools/releases/download/v1.0.0/Thermal_Tools.AppImage

chmod +x ./Thermal_Tools.AppImage && ./Thermal_Tools.AppImage