immich
immich copied to clipboard
immich-app
Exif date/time isn't parsed correctly
The bug
(Note this might be related to / same as #4898)
We have two different models of phones,
- Fairphone 4, CalyxOS, OpenCamera v1.52 (mine)
- Samsung Galaxy Note 10, Samsung Android (?), Samsung Camera v? (wifey's)
Both cameras sync to Immich server but our photos are not in order and I started investigating.
My photos are saved with wrong date/timestamp. I'm currently in UTC+2 (Stockholm/Europe). Comparing EXIFs (exiftool 12.76):
Photos
Fairphone 4:
File Name : IMG_20240504_164142.jpg
File Modification Date/Time : 2024:05:04 16:41:42+02:00
File Access Date/Time : 2024:05:06 19:05:53+02:00 *
File Inode Change Date/Time : 2024:05:06 19:06:02+02:00 *
Date/Time Original : 2024:05:04 16:41:42
Create Date : 2024:05:04 16:41:42
Create Date : 2024:05:04 16:41:42.809319
Date/Time Original : 2024:05:04 16:41:42.809319
Modify Date : 2024:05:04 16:41:42.809319
* <- when I syncthing'd it
This Date/Time is translated to
05/04/2024, 06:41 PM (18:41)
Africa/Blantyre (+02:00)
Wrong time, wrong timezone ...
Samsung Galaxy Note 10:
File Name : 20240504_153307.jpg
File Modification Date/Time : 2024:05:04 15:33:09+02:00
File Access Date/Time : 2024:05:04 15:33:09+02:00
File Inode Change Date/Time : 2024:05:06 20:02:35+02:00
Date/Time Original : 2024:05:04 15:33:08
Create Date : 2024:05:04 15:33:08
Create Date <-- SGN10 doesn't have this at this pos
Date/Time Original : 2024:05:04 15:33:08+02:00
Modify Date : 2024:05:04 15:33:08+02:00
This Date/Time is translated to
05/04/2024, 03:33 PM (15:33)
Africa/Blantyre (+02:00)
Correct time, wrong timezone ...
Videos
Fairphone 4:
File Name : VID_20240504_164249.mp4
File Modification Date/Time : 2024:05:04 16:44:35+02:00
File Access Date/Time : 2024:05:06 19:06:24+02:00 *
File Inode Change Date/Time : 2024:05:06 19:06:34+02:00 *
Create Date : 2024:05:04 14:44:35
Modify Date : 2024:05:04 14:44:35
Track Create Date : 2024:05:04 14:44:35
Track Modify Date : 2024:05:04 14:44:35
Media Create Date : 2024:05:04 14:44:35
Media Modify Date : 2024:05:04 14:44:35
* <- when I syncthing'd it
This Date/Time is translated to
05/04/2024, 02:44 PM (14:44)
Africa/Abidjan (+00:00)
Which kind of translates to "correct" even if the correct should be "05/04/2024, 04:44 PM (16:44), Europe/Stockholm (+02:00)"
Samsung Galaxy Note 10:
File Name : 20240504_154055.mp4
File Modification Date/Time : 2024:05:04 15:41:04+02:00
File Access Date/Time : 2024:05:04 15:41:04+02:00
File Inode Change Date/Time : 2024:05:06 20:02:52+02:00
Create Date : 2024:05:04 13:41:04
Modify Date : 2024:05:04 13:41:04
Track Create Date : 2024:05:04 13:41:04
Track Modify Date : 2024:05:04 13:41:04
Media Create Date : 2024:05:04 13:41:04
Media Modify Date : 2024:05:04 13:41:04
This Date/Time is translated to
05/04/2024, 03:41 PM (15:41)
Africa/Blantyre (+02:00)
Correct time, wrong timezone ...
From https://web.archive.org/web/20180921145139if_/http://www.cipa.jp:80/std/documents/e/DC-010-2017_E.pdf one can read:
- DateTimeOriginal, DateTimeDigitized - The date and time when the image was stored as digital data (no time zone in Exif), stored in ISO 8601 format, not in the original Exif format. This property includes the value for the Exif SubSecTimeDigitized attribute.
From https://en.wikipedia.org/wiki/ISO_8601 on can read:
- Time zones in ISO 8601 are represented as local time (with the location unspecified), as UTC, or as an offset from UTC. Time zone designators:
<time>Z <time>±hh:mm <time>±hhmm <time>±hh - If no UTC relation information is given with a time representation, the time is assumed to be in local time.
Now, this is clearly not what's happening in Fairphone 4 case above:
- "2024:05:04 16:41:42.809319" is being translated into "05/04/2024, 06:41 PM (18:41) Africa/Blantyre (+02:00)".
- The date/time is already in UTC+2 ("local time").
Something is clearly not being parsed correctly somewhere .. and I have thousands of photos that are sorted wrongly in the archive now ..
Please advice ..
The OS that Immich Server is running on
Debian
Version of Immich Server
v1.99.0
Version of Immich Mobile App
1.100.0 build.130
Platform with the issue
- [X] Server
- [X] Web
- [ ] Mobile
Your docker-compose.yml content
N/A .. running podman
podman pod create --name pod-immich --hostname immich --publish 2283:3001
podman run -d --pod pod-immich --name immich_redis redis:6.2-alpine
podman run -d --pod pod-immich --name immich_postgres --env-file $DP/immich.env --volume $DP/postgres:/var/lib/postgresql/data tensorchord/pgvecto-rs:pg14-v0.2.0
podman run -d --pod pod-immich --name immich_machine_learning --env-file $DP/immich.env --volume $DP/model-cache:/cache immich-machine-learning:v1.99.0
podman run -d --pod pod-immich --name immich_server --env-file $DP/immich.env --volume $DP/upload:/usr/src/app/upload --volume /etc/localtime:/etc/localtime:ro immich-server:v1.99.0 start.sh immich
podman run -d --pod pod-immich --name immich_microservices --env-file $DP/immich.env --volume $DP/upload:/usr/src/app/upload --volume /etc/localtime:/etc/localtime:ro immich-server:v1.99.0 start.sh microservices
Your .env content
# General
TZ="Europe/Stockholm"
# Database
DB_HOSTNAME=immich_postgres
DB_USERNAME=immich
DB_PASSWORD=xxxxx
DB_DATABASE=immich
DB_DATABASE_NAME=immich
# Postgres
POSTGRES_USER=immich
POSTGRES_PASSWORD=xxxxx
POSTGRES_DB=immich
# Redis
REDIS_HOSTNAME=immich_redis
Reproduction steps
N/A
Relevant log output
No response
Additional information
No response
I'm seeing similar problems on my GrapheneOS phone as well. Seems to impact videos and photos differently. Sometimes tagged with local time zone, other times, UTC. -- but odd, because it's not just the wrong time zone, it can be the wrong day in UTC. IE videos I took last night are saying they're in the future -- for tonight.
I see similar issues on my end. In my case I my pictures where saved into iCloud Photos from a DJI Osmo 3, and when the app imported them into Immich the datetime was wrong by 2 hours.
I removed all pictures and tried to import them manually via the web interface, same result.
Attaching one of the pictures that has that issue.
Yeah most of my issues result from my Osmo Pocket 3 as well. My guess is this is because the photos lack GPS data, but it would be nice if it still had accurate time.
One thing I considered is that I tend to rename the lrf files to MP4 so I can upload those, and I keep the full size media elsewhere for editing. Perhaps DJI does something weird with the LRF exif.
-------- Original Message -------- On 5/12/24 2:31 AM, Felipe Martin wrote:
I see similar issues on my end. In my case I my pictures where saved into iCloud Photos from a DJI Osmo 3, and when the app imported them into Immich the datetime was wrong by 2 hours.
I removed all pictures and tried to import them manually via the web interface, same result.
Attaching one of the pictures that has that issue.
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>
Chiming in; same behavior here with older photos shot on a Motorola Moto G (1. Gen), additionaly it dates it far back (photo from 2016, Immich dates it as 2002) For comparison, Windows Explorer seems to parse the information correctly. I have my external library bound via a read only share, might this be a cause for the metadata not being processed correctly? (I am aware that manual changes to the metadata to these files cannot be saved, as it has no write access on the source folder)
EDIT: Okay, I just learned that Windows Explorer doesn't always read EXIF data.
In my case, windows explorer read the base file attribute but there ACTUALLY IS a "Created"-Date with an actual 2002-date for some reason (inspected with ExifTool). Hopefully that helps some people further diagnosing their metadata ✌️
Was at the time of reporting, yes .. .. and it's unfortunately a bit of a hassle to update so I refrained from doing so until had some feedback in this thread :-)
I am pretty sure the latest version was v1.105.0 two weeks ago. Can you help upgrade it and test it again? Also include the troubled file will help with troubleshooting as well.
I just updated to v1.105.1 and refreshed the metadata for my entire library, still having the same issue. Sample file in my previous comment.
Just updated to
- App version 1.105.1 build.140 and
- Server version 1.105.1
Tested and it is still borked ..
Attached sample file ...
That becomes like follows in Immich ...
@ygaeon The timezone is parsed based on a few heuristics, so it isn't easy to explain precisely which one is used. From my observation in the past, CalyxOS strips GPS data, which the parsing mechanism uses to identify the correct timezone
You can read more here https://photostructure.github.io/exiftool-vendored.js/#md:dates
Thanks @alextran1502 and yes, CalyxOS do but I'm using "OpenCamera" which does not strip GPS, afaik. Though, I fail to see why GPS would be needed in this case as the exif data is present. Please just use it when it is there?
$ identify -verbose IMG_20240605_002021.jpg
Image:
Filename: IMG_20240605_002021.jpg
...
exif:DateTime: 2024:06:05 00:20:20
exif:DateTimeDigitized: 2024:06:05 00:20:20
exif:DateTimeOriginal: 2024:06:05 00:20:20
...
I checked the attached file, following your link @alextran1502 ...
Heuristic 1: explicit metadata
- My image doesn't have
TimeZoneOffset -
exif:DateTimeOriginal: 2024:06:05 00:20:20is set -
ModifyDateis NOT set - None of
OffsetTime,OffsetTimeOriginal, norOffsetTimeDigitizedis set.
Heuristic 2: GPS location
No GPS location available
Heuristic 3: UTC timestamps
- Neither
GPSDateTimenorDateTimeUTCis set.
.. so I guess that exiftool-vendored would assume UTC in this case .. making all photos off by the 1-2 hours I see ..
In this case, would it not be possible to "attach" the timezone (from the mobile device) to the image/video being uploaded to the store?
Opened this over 4 months ago, still have the same issues. Anyone more knowledgeable have any comments or feedback?
Server Version: v1.114.0 App Version: 1.114.0 build.158
Some observations
Starting info:
- Our image has dateTimeOriginal
- Does not have GPS or any exif tags that represent the timezone
The way I observe immich working when it does not have any information related to timezone:
- Assume that the time (Date Time Original or whatever it has) is in UTC
- But uses the timezone of the server (set by an environment variable in the docker compose). Hence you see that GMT + 2:00. If the time in UTC was 00:20, then in the timezone the image was taken in, the local time was 02:20. Note: The browser and the app always displays the local time of wherever the image was taken. Hence, in the browser you see 02:20 GMT +2:00.
What would likely be an expected behavior is this:
- Assume a timezone for that image
- Then assume that the time (dateTimeOriginal or whatever the image has) is also in the same timezone it assumed in step 1
- If you assumed timezone as UTC in step 1 --> Also assume that the dateTimeOriginal is in UTC
- If you assumed timezone is the timezone of the server --> Also assume the dateTimeOriginal in server timezone. This is the better option in my opinion.
Many cameras without GPS just store DateTimeOriginal tags without timezone information. We often take pictures while on vacation in timezones that are not the same as the immich server and not in UTC. I don't think immich should ever assume what the timezone is. If it can't be determined from the image metadata, then the timezone should be explicitly stated by the user during import. This is only necessary because immich likes to attach a timezone to timestamps it stores. Other photo management systems do not care about timezone and the assumption is that the DateTimeOriginal is simply in whatever timezone where the photo was captured.
Correction to my previous comment - by timezone of the server, I actually meant the timezone the user sets using a env variable in the compose.
Also, I dont think that there is a solution without having a timezone attached to an image. I mean the entire concept of time is relative. So how would any application be able to provide a timeline view of images that can be taken in multiple timezones. Everything needs to be brought to a common reference, which is UTC, and we need the timezone to do it.
I agree it is nice to have the proper timezone attached to an image, but it was not deemed necessary when EXIF metadata was invented for use with digital cameras. I've been using the photo management site flickr for 20 years with images that don't have timezone information and the timeline features work just fine when you understand the time displayed is wherever the photo was captured. Flickr and other photo sites do not assume a timezone. Only the photographer knows the correct timezone in these cases so we need a way to pass that to immich during photo import if immich insists on storing timezone. If the photographer travels often, it can't be a fixed offset set from an env variable.
That is confusing. Maybe getting a bit off topic here - I still do not think flicker's timeline would work well ( it can still work. Bit is the timeline accurate - dont think so). What happen's if you upload 2 images. A - 10 AM (no timezone information in the image. But it was actually taken in PST) B - 11 AM EST.
Which image would come first in the timeline in Flicker? Flicker would need to assume a timezone for the images that dont have timezone OR just have B (11 am) come after A (10 am) just because of the time, which could most certainly be wrong.
Coming back to the original problem :), Immich does have environment variable that set the timezone of images that do not have a timezone. but it might also be worthwhile to be able to edit the timezone in the GUI i think.
It is a known drawback of EXIF that until recently it did not provide a way for cameras to record time-zone information. Any long-lived photo management system has had to deal with this. Flickr and others simply do not use timezone information for the timeline since historically it has not been available. Digital cameras existed for 20 years before geotagging became common and before the EXIF spec was updated to allow timezone with OffsetTimeOriginal. If you upload two photos to flickr, one with timezone data and one without, it will ignore the timezone and make no assumptions about timezone for the other. The timeline will still be correct and ordered exactly as captured. It would only be a problem if your timeline somehow contains photos captured by multiple photographers shooting simultaneously in different timezones.
Thanks for some activity in this issue .. :-)
@Sean-T-Moore : If it can't be determined from the image metadata, then the timezone should be explicitly stated by the user during import.
Well, if the timezone is missing in the metadata I think the closest truth would be the date and time and timezone set on the mobile device .. I mean, we usually update the time on our mobile device when we travel, right?
@ygaeon: I think the closest truth would be the date and time and timezone set on the mobile device
I import photos when I get home, not when I'm away in a different time zone. So mobile device timezone can't be used to set image metadata during import. Mobile devices really should be smart enough to set image metadata correctly already during image capture.
Maybe a section in the UI showing the latest import along files already stored with the option to modify them in batch, to check if everything is set ok or modify datetime settings accordingly.
Issues with parsing date/time incorrectly should go to #12650, potential feature requests (bulk modify assets) should be a feature request (pretty sure there already is one anyways)
