Fails to process JPEG from raw DNG (workaround included)

[blaueente]

Basic information

• mapillary_tools version 0.10.1
• System: Linux
• Capture Device: Android with Open Camera, Raw DNG images, edited with darktable

Steps to reproduce behavior

1. take .dng raw image
2. process with darktable into .jpg
3. add geotags
4. upload to mapillary

Expected behavior

upload success

Actual behavior

Fails to upload with error:

2023-10-02 14:56:30,410 - WARNING - Unknown error test writing image IMG_20230922_165457.jpg
Traceback (most recent call last):
  File "/home/mapuser/.local/lib/python3.11/site-packages/mapillary_tools/exif_write.py", line 151, in _safe_dump
    exif_bytes = piexif.dump(self._ef)
                 ^^^^^^^^^^^^^^^^^^^^^
  File "/home/mapuser/.local/lib/python3.11/site-packages/piexif/_dump.py", line 63, in dump
    zeroth_set = _dict_to_bytes(zeroth_ifd, "0th", 0)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mapuser/.local/lib/python3.11/site-packages/piexif/_dump.py", line 335, in _dict_to_bytes
    length_str, value_str, four_bytes_over = _value_to_bytes(raw_value,
                                             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mapuser/.local/lib/python3.11/site-packages/piexif/_dump.py", line 205, in _value_to_bytes
    four_bytes_over = _pack_short(*raw_value)
                      ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mapuser/.local/lib/python3.11/site-packages/piexif/_dump.py", line 168, in _pack_short
    return struct.pack(">" + "H" * len(args), *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
struct.error: required argument is not an integer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mapuser/.local/lib/python3.11/site-packages/mapillary_tools/process_geotag_properties.py", line 297, in _verify_image_exif_write
    edit.dump_image_bytes()
  File "/home/mapuser/.local/lib/python3.11/site-packages/mapillary_tools/exif_write.py", line 194, in dump_image_bytes
    exif_bytes = self._safe_dump()
                 ^^^^^^^^^^^^^^^^^
  File "/home/mapuser/.local/lib/python3.11/site-packages/mapillary_tools/exif_write.py", line 152, in _safe_dump
    except struct.error:
           ^^^^^^
NameError: name 'struct' is not defined

Corresponding data

Additional information

This is related to a piexif bug https://github.com/hMatoba/Piexif/issues/86

This should be fixed directly in piexif, but for a workaround I quickly patched the upload script:

mapillary_tools/exif_write.py", line 152, in _safe_dump :

        while True:
            try:
                exif_bytes = piexif.dump(self._ef)
# add this clause here: 
            except struct.error:
                del self._ef["0th"][piexif.ImageIFD.AsShotNeutral]
                LOG.debug("Workaround: deleted AsAshotNeutral Tag")
# until here
            except piexif.InvalidImageDataError as exc:
                if thumbnail_removed:
                    raise exc
                LOG.debug(
                    "InvalidImageDataError on dumping -- removing thumbnail and 1st: %s",
                    exc,
                )
                # workaround: https://github.com/hMatoba/Piexif/issues/30