HEIF images cannot be previewed in the PC client.

[imysl0]

• Operating System (Windows):
• Delta Chat Version:1.44.1
• Expected behavior: PNG images can be previewed
• Actual behavior: PNG images cannot be previewed
• Steps to reproduce the problem: The issue consistently arises when sending the image without compression.
• Screenshots:

The files are here. To prevent the website from automatically compressing the images, I've also provided a zip archive.

png：

zip ：

image_2024-05-29_08-35-40.zip

[imysl0]

The file displays and previews correctly on both mobile clients and Windows systems.

[maxphilippov]

@imysl0 can you provide any information on how you made this image? Cause I'm having trouble opening this one with anything on Windows (neither Explorer, Chrome, Firefox, Python IPL, nor Windows Photos work)

[imysl0]

@maxphilippov

• Although its suffix is .png, I suspect it's actually a HEIF image file.

• My friend sent this to me, and he most likely downloaded it from the Chinese version of TikTok, which is called Douyin, or from Xiaohongshu (Little Red Book). Owing to the explosion and chaos of information flow, the original download page has been lost in the river of time, rendering it irrecoverable and beyond tracing.

• On my Windows 11 PC, I can open it without any special action; the system previews it directly or allows me to open it with the built-in Paint app. It's quite puzzling. After searching online, it might be related to my having installed the HEIF Image Extensions (though I don't recall doing so).

[maxphilippov]

Thanks for clarifying this! I didn't manage to make it work (I got HEIF extension installed, but there's a chance it HEIC format which requires HEVC codec extension which I can't install right now, I know, I know it's confusing a bit).

You're on Win11, that means you got it installed by default (since 2H22 or whatever they call it).

I've renamed the issue if you don't mind. Just one last question: can you try to download it from the dialog and check if it opens?

HEIF support will probably require deltachat-core implementation @Simon-Laux, @r10s and probably adding another library? I don't know if it's actually worth it to guess format by file header in binary when uploading from UX standpoint.

[iequidoo]

The image crate used by Core doesn't support HEIF, so the first problem is that we can't recode it to reduce its size as for other formats. The second one is that apparently not all platforms support HEIF, that's why currently we use Viewtype::File for it, not Viewtype::Image. In this issue the image went to the "Images" tab only because it has ".png" extension. But maybe it's not a big problem if HEIF will be Image and just not displayed correctly? This looks not nice, but having images in the "Files" tab is not much better, moreover on platforms that are able to display them.

[imysl0]

can you try to download it from the dialog and check if it opens?

• I saved this image from the chat window (even though it couldn't be displayed), and then on a Windows system, it opened normally with no change in size. @maxphilippov
• It seems this situation is indeed rare; whether to address it is up to you. It's not urgent or necessary. Fine-tuning details might be better handled when there are more resources available.
• Thank you very much for your response and for providing patient explanations.

[WofWca]

I suggest to either transfer this to Core (as per the comment https://github.com/deltachat/deltachat-desktop/issues/3879#issuecomment-2206741104), or close as "not planned".

I think we can't do much about whether the browser engine supports certain codecs. Apparently there are no plans to support HEIF in Chromium: https://groups.google.com/a/chromium.org/g/chromium-os-dev/c/AxaBskFfoho Here is a page that shows support for HEIF: https://www.lambdatest.com/web-technologies/heif-chrome. None currently support it, according to it.

[Simon-Laux]

On the other hand afaik it is the default format that the iOS camera produces, so quite common.

[r10s]

On the other hand afaik it is the default format that the iOS camera produces, so quite common.

that's true - but using the default flow, these HEIF images do not land in Delta Chat - if you select a HEIF from the gallery or use the camera directly, it will still be sent as JPG (i just tested that)

still, it may happen by other flows, and it is not that uncommon (also android supports it since android 10)

what we can do as a minimum, however, is to improve the error handling:

• show a placehodler that says "View image with an external Viewer"
• once that text is tapped, call the system's "open" with the file, assuming some file viewer will be able to display it (we do sth. similar on iOS when receiving weird video formats, not perfect, but that's it)

EDIT: there is https://alexcorvi.github.io/heic2any/ - but not sure if that is doable and worth the effort, looks quite unmaintained and will be another can of worms. i'd go with the external viewer. that may also be a pattern for similar things

[Simon-Laux]

Android seems to have support for HEIF/HEIC since android 10, https://developer.android.com/reference/android/graphics/ImageFormat#HEIC (but some docs site said it needs a hardware encoder).

There are libraries in rust, but all I found are wrappers around https://github.com/strukturag/libheif, which is written in C++.

Generally I'm a bit sceptical about media decoders for new formats that are not written in rust because of the webp CVE that was affecting everyone a while ago.

For desktop we could use js libraries and or wasm libraries:

• the one bjoern mentioned: Heic2any
• https://www.npmjs.com/package/libheif-js
• wasm https://github.com/hpp2334/elheif
• wasm demo https://strukturag.github.io/libheif/ (this blog post says it is ~2.5mb: https://shkspr.mobi/blog/2020/12/coping-with-heic-in-the-browser/)

If we invest the time for adding a placeholder, then we would also be able to try the wasm library. but there should be a time limit on trying this with a fallback to just doing the placeholder.

[Simon-Laux]

nowadays it is shown as file, and if you rename it to png then you can not send it as image. delta tauri would support heic images, but only on macOS.

Support for this edge case is out of scope, so marking this as won't fix.

[Simon-Laux]

well sending it with wrong png file extension as file still sends it.

I think we need a better detection of filetype. that is not only based on extension.

[Simon-Laux]

actual issue seems to be send as file with a heic file that was renamed to have the png extension. So our file type detection is at fault here for giving it the png mimetype, we should not only rely on mimetype for that: there is a rust crate that is also used by tauri, which can infer the filetype from the magic numbers of the files content: https://lib.rs/crates/infer - maybe we could add that into core, so it doesn't tell desktop the wrong mimetype in Message.file_mime?

[iequidoo]

Are renamed files with changed extensions a frequent problem? Even if we use this infer crate, Core may return a wrong mimetype for attachments in received messages because it prefers to look into Content-Type header. We can change this logic as well though.

[Simon-Laux]

This is mostly about sending, I think for receiving it seems fine to use what content-type provides

[trufae]

i think for security and performance reasons, the sender must encode the image into jpg unless its send as a raw file. there are no decent heif implementations and the standard is very complex

[iequidoo]

i think for security and performance reasons, the sender must encode the image into jpg unless its send as a raw file. there are no decent heif implementations and the standard is very complex

At the moment the core lib doesn't even support decoding HEIF because image crate doesn't support that. So we can't do much here. Otherwise i agree that it's better to always recode HEIF to JPEG as the former isn't supported well on all platforms. We need to look for alternative crates, but anyway switching to another one is some work as the core uses a bunch of image interfaces currently.

[Simon-Laux]

at this point the thing is that I want to avoid the case where the user thinks the bug is in DC because the image is not loaded. This particular issue is user error, because they drag in a malformed file with a wrong file ending. Nonetheless currently it looks like dc is buggy, because it attempts to load the image, but fails.

One solution could be to not only use the file ending for type detection as I wrote about in https://github.com/deltachat/deltachat-desktop/issues/3879#issuecomment-2770509529

But since there is so much confusion in this issue thread, we may open a new one (with better issue description) and close this one as won't fix.

[iequidoo]

We already have image format guessing in Core's BlobObject::recode_to_size(), but if it fails, it just proceeds with the original "image". In fact we don't know if it's an image then, so we can just change its viewtype to File. EDIT: This may be better than guessing based on magic numbers in the file content, presense of magic numbers doesn't guarantee that the image isn't corrupted.

[iequidoo]

https://github.com/chatmail/core/pull/6904 is merged, so now unrecognized images are sent as Files, they are only renamed to "image_DATETIME.EXT" to not reveal the original file name. If such an image contains Exif, sending will fail. The user still can send it as File or remove Exif by hand and send it as Image (we don't send images with Exif for privacy reasons).