Bug - XMP sidecar timestamp may be a second or two ahead of the database timestamp

[Ashley-Butcher]

Describe the bug

The XMP sidecar timestamp may be a second or two ahead of the database timestamp, resulting in a prompt every time you open Darktable to update from XMP sidecar files.

Steps to reproduce

1. Perform an action on many images to generate/update several XMP sidecar files (i.e. add a tag to 500-1000 images)
2. Close Darktable
3. Open Darktable

(This issue occurs with slower edits on just one image, but it's easier to reproduce with a larger number of images).

Observe: A prompt to update from sidecar file will appear with a timestamp delta of 1-2 seconds.

Expected behavior

If the XMP sidecar file is not updated outside of Darktable, no "update" should be found - i.e. the timestamp in the database should be the same as the file timestamp.

Logfile | Screenshot | Screencast

Commit

No response

Where did you install darktable from?

darktable.org

darktable version

4.4.2

What OS are you using?

Windows

What is the version of your OS?

Windows 11 Pro 22H2 (build 22621.2428)

Describe your system?

No response

Are you using OpenCL GPU in darktable?

Yes

If yes, what is the GPU card and driver?

NVIDIA GeForce RTX 3090, 24 GiB RAM dedicated (90 GiB shared), version 537.58

Please provide additional context if applicable. You can attach files too, but might need to rename to .txt or .zip

I've checked with Sysinternals Process Monitor (procmon) to ensure no other application is updating the timestamp on the XMP sidecar files in-between closing Darktable and reopening.

This is more likely to occur (and easier to reproduce) when modifying images in bulk, which is why this is the method given in the reproduction steps.

[parafin]

Sounds a bit like #12257, but not sure if they are the same. Which filesystem is that? As far as I remember FAT filesystems have only 2-second precision of timestamps. But I see that reported timestamps are both odd and even, so it’s not that either…

[Ashley-Butcher]

@parafin I saw #12257, also #13679 and #13298, so I'm sorry if it's a duplicate, but it didn't seem like an exact match, and it's still a real problem.

I'm getting this on large edits on a local NTFS formatted drive, but also on network shares via SMBv3. For workflow reasons I'm doing this part of my PP in Windows, and I know there are a lot of ways for timestamp resolution to be an issue here, but then again I think then there's a flaw somewhere in how the crawler detects changes.

Normally NTFS doesn't have such a precision issue. SMB on the other hand, not sure, as there are many factors that could reduce the precision. Some cloud storage synchronising software doesn't do nice things to timestamps either.

Maybe there should be a hash of the XMP file in the sqlite library to test as a secondary check if the timestamp is within a certain threshold, such as ≲ 2 seconds to cover timestamp resolution issues?

I don't know enough about the inner workings of Darktable to comment, though.

[github-actions[bot]]

This issue has been marked as stale due to inactivity for the last 60 days. It will be automatically closed in 300 days if no update occurs. Please check if the master branch has fixed it and report again or close the issue.

[github-actions[bot]]

This issue has been marked as stale due to inactivity for the last 60 days. It will be automatically closed in 300 days if no update occurs. Please check if the master branch has fixed it and report again or close the issue.