"Cannot sync due to invalid modification time"

[ThatCoffeeGuy]

How to use GitHub

• Please use the 👍 reaction to show that you are affected by the same issue.
• Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
• Subscribe to receive notifications on status change and new comments.

Expected behaviour

It should be able to sync files with no issues

Actual behaviour

Complains about invalid modification time while basically everything else works perfectly with the same files

Steps to reproduce

1. Fill up your nextcloud instance with terabytes of files
2. Wait a few months, it eventually starts corrupting your files

Client configuration

Version 3.4.4 (Windows).

Operating system: Windows 10

OS language: US ENG

Qt version used by client package (Linux only, see also Settings dialog):

Client package (From Nextcloud or distro) (Linux only):

Installation path of client: default

Server version

22.2.5

I can see that it had ruined many people's files back in 2021 https://syncloud.discourse.group/t/disaster-with-nexcloud-3-4-0-client/141/4

But I am not sure if I was using the same instance back then. It seems this issue is still present. I am spending hours on trying to move files, rescan them, whatever. It is really making me stop recommending this (potentially) amazing product to anyone.

[liquidbase]

Identical problem with Windows 11.

Client version is current 3.4.4 Server version is currently 23.0.3

Trying to fix the problem by setting the date of the last modification via Linux command touch to the current date does not interest the client. Even after a clean reinstallation of the client, the sync errors reappear right after the first sync.

Is there a possibility that the sync errors are stored in the database, preventing the client from starting a proper sync?

edit A manual fix of the sync is possible depending on the size of the Nextcloud and would work as follows:

1. Find all files with wrong modification date with the command find /path/to/data/directory/ -type f -name "*" -newermt 1970-01-01 ! -newermt 1970-01-02
2. Scan all files to correct the database values with the command sudo -u www-data php occ files:scan --all
3. Restart sync

If the sync continues to generate errors, you will need to manually clean up the user's deleted files on the server.

[hwangjunjie]

Thanks, this method works and I have fixed it. Here is the full command: find /path/to/data/directory/ -type f -name "*" -newermt 1970-01-01 ! -newermt 1970-01-02 -print0 | xargs -0 touch -m

[Sieboldianus]

nextcloud-client: 3.4.4 server: 22.2.5

Saw this error today on my main Documents folder sync, the first time after using Nextcloud 3 years (folder size: 40GB, quite a few files).

find /nectloud/server/data -type f -name "*" -newermt 1970-01-01 ! -newermt 1970-01-02

gives no results.

Are there any instructions/steps what else could be done/how to solve this issue?

Inside Documents, all files show green checkmarks and it doesn't say which files this applies to. If I right-click on the error in NC client and on "View activity", I get an empty window.

[Edit] I can see that the Documents folder in the Web-App shows a wrong date (modified in the future, 86 years from now):

On the server drive (ZFS), the folder has the correct date:

ls -alh --full-time
drwxr-xr-x 39 www-data www-data 49 2021-11-22 08:31:30.803407858 +0100  Documents

• Rescanning the whole folder (sudo -u www-data php occ files:scan --path "user/files/Documents") does not work (error persists), same with files:scan --all
• Deleting trash on the server for this user does not work

Atm. I assume this happened because I deleted a single photo through the web app today. I usually never do this (always remove the files on harddisk and let it sync). However, I don't know why this resulted in a broken timestamp on Nextcloud's database, and I don't know why it cannot get the correct one from the filesystem, since it is all there and correct.

[Edit2]

Also tested solvable_files.sh with list argument, but no output/no files detected.

[Edit3]

No way to solve this error with the sh files provided by Nextcloud so far.

Leaving this at it today, perhaps tomorrow someone will find a solution.

Here're the log lines from Desktop Client that appear to point to the issue:

click

2022-03-24 18:43:48:692 [ info nextcloud.sync.propagator libsync\owncloudpropagator.h:212 ]:	Starting CSyncEnums::CSYNC_INSTRUCTION_ERROR propagation of "Documents" by OCC::PropagateIgnoreJob(0x19f124febe0)
2022-03-24 18:43:48:692 [ debug nextcloud.sync.database.sql common/ownsql.h:145 ]	[ OCC::SqlQuery::bindValue ]:	SQL bind 1 "Documents"
2022-03-24 18:43:48:692 [ warning nextcloud.sync.propagator libsync\owncloudpropagator.cpp:281 ]:	Could not complete propagation of "Documents" by OCC::PropagateIgnoreJob(0x19f124febe0) with status OCC::SyncFileItem::NormalError and error: "Cannot sync due to invalid modification time"
2022-03-24 18:43:48:692 [ debug nextcloud.sync.statustracker libsync\syncfilestatustracker.cpp:274 ]	[ OCC::SyncFileStatusTracker::slotItemCompleted ]:	Item completed "Documents" OCC::SyncFileItem::NormalError CSyncEnums::CSYNC_INSTRUCTION_ERROR
2022-03-24 18:43:48:692 [ warning nextcloud.gui.activity gui\tray\usermodel.cpp:561 ]:	Item  "Documents"  retrieved resulted in  "Cannot sync due to invalid modification time"
2022-03-24 18:43:48:692 [ warning nextcloud.gui.activity gui\tray\usermodel.cpp:538 ]:	Item  "Documents"  retrieved resulted in error  "Cannot sync due to invalid modification time"
2022-03-24 18:43:48:692 [ info nextcloud.gui.activity gui\tray\activitylistmodel.cpp:402 ]:	Error successfully added to the notification list:  "Cannot sync due to invalid modification time"
2022-03-24 18:43:48:692 [ info nextcloud.sync.propagator libsync\owncloudpropagator.cpp:1179 ]:	PropagateDirectory::slotFirstJobFinished emit finished OCC::SyncFileItem::NormalError
2022-03-24 18:43:48:692 [ info nextcloud.sync.propagator.root.directory libsync\owncloudpropagator.cpp:1306 ]:	OCC::SyncFileItem::NormalError slotSubJobsFinished OCC::PropagatorJob::Running pending uploads 0 subjobs state OCC::PropagatorJob::Finished
2022-03-24 18:43:48:692 [ info nextcloud.sync.propagator libsync\owncloudpropagator.cpp:1320 ]:	PropagateRootDirectory::slotSubJobsFinished emit finished OCC::SyncFileItem::NormalError
2022-03-24 18:43:48:694 [ debug nextcloud.sync.database.sql common\ownsql.cpp:295 ]	[ OCC::SqlQuery::exec ]:	SQL exec "SELECT path FROM conflicts"
2022-03-24 18:43:48:694 [ debug nextcloud.sync.database.sql common\ownsql.cpp:295 ]	[ OCC::SqlQuery::exec ]:	SQL exec "DELETE FROM flags WHERE path != '' AND path NOT IN (SELECT path from metadata);"
2022-03-24 18:43:48:694 [ debug nextcloud.sync.database.sql common\ownsql.cpp:328 ]	[ OCC::SqlQuery::exec ]:	Last exec affected 0 rows.
2022-03-24 18:43:48:695 [ warning nextcloud.gui.folder gui\folder.cpp:971 ]:	SyncEngine finished with ERROR

[Sieboldianus]

Solved it for the moment:

choco uninstall nextcloud-client
choco install nextcloud-client --allow-downgrade --version 3.4.3

Now the desktop client syncs again and does not error on Modification date. The wrong (future) modification date on Documents folder is still there in the Nexctloud App/Database, but at least I can sync again. This is a workaround, not a solution.

[AndyXheli]

@Sieboldianus

Hi. That’s how I managed to solve this

First of all you need to find all the files with destroyed timestamp

find yournextcloudfilelocation/ ! -newermt 1970-01-02 > damaged.txt then use this script to touch and modify the date. You can set whatewer the date you want just change 15 jan

#!/bin/bash

file="damaged.tx"

while read -r line; do touch -d '15 jan' "$line" done <$file

Then If you use docker

sudo docker exec --user www-data your containerID php occ files:scan --all if you don’t use docker then

sudo -u www-data php occ files:scan --all

after that you need to sync all your clients from scratch

https://help.nextcloud.com/t/desktop-client-3-4-0-destroys-local-time-stamp-and-keeps-uploading-data-to-server/128512/96

[Sieboldianus]

@andyxheli Thanks andy. I tested these steps. find yournextcloudfilelocation/ ! -newermt 1970-01-02 > damaged.txt and similar commands did not return any files, same with the Nextcloud provided files.

There was only one folder affected: My main Documents folder, which had a date modified set to 2107 (86 years from now in the future). On the filesystem (server), the date was correct. On the client, the date was also correct.

:warning: The wrong date was only in the nextcloud database. :warning:

Here is how I solved this (just now).

cd /var/www/nextcloud/data/user/files/
touch -m Documents # Update only the last modified date to today
occ files:scan --path "user/files/Documents"' # scan again

tada:

What wasn't clear to me is that files:scan will only pick up files that changed.

Also updated nextcloud-client to 3.4.4 again and can confirm that the error disappeared.

[exchange12rocks]

Also if you have a Windows client, you can fix broken files from the client-side with this PowerShell one-liner (run it in the NextCloud folder in your user profile):

Get-ChildItem -Recurse | Where-Object -FilterScript {$_.LastWriteTime -eq ([datetime]::new(621355968000000000))} | ForEach-Object -Process {$_.LastWriteTime = Get-Date}

[Sieboldianus]

Also if you have a Windows client, you can fix broken files from the client-side with this PowerShell one-liner (run it in the NextCloud folder in your user profile):

Get-ChildItem -Recurse | Where-Object -FilterScript {$_.LastWriteTime -eq ([datetime]::new(621355968000000000))} | ForEach-Object -Process {$_.LastWriteTime = Get-Date}

I doubt this would work, since the client refused to sync to the server (at all) - because the wrong date was on the server (database only, fs not affected), not on the client side.

[exchange12rocks]

Also if you have a Windows client, you can fix broken files from the client-side with this PowerShell one-liner (run it in the NextCloud folder in your user profile):

Get-ChildItem -Recurse | Where-Object -FilterScript {$_.LastWriteTime -eq ([datetime]::new(621355968000000000))} | ForEach-Object -Process {$_.LastWriteTime = Get-Date}

I doubt this would work, since the client refused to sync to the server (at all) - because the wrong date was on the server (database only, fs not affected), not on the client side.

Works for me, mate 🤷‍♂️ - all green and happy now

[Maxplosion]

Also if you have a Windows client, you can fix broken files from the client-side with this PowerShell one-liner (run it in the NextCloud folder in your user profile):

Get-ChildItem -Recurse | Where-Object -FilterScript {$_.LastWriteTime -eq ([datetime]::new(621355968000000000))} | ForEach-Object -Process {$_.LastWriteTime = Get-Date}

Thanks, that worked for me to adjust the filedates, just had to change the timestamp to [datetime]::new(621356004000000000) as mine were 01.01.1970 01:00 instead of 00:00.

[pwnorbitals]

Also if you have a Windows client, you can fix broken files from the client-side with this PowerShell one-liner (run it in the NextCloud folder in your user profile):

Get-ChildItem -Recurse | Where-Object -FilterScript {$_.LastWriteTime -eq ([datetime]::new(621355968000000000))} | ForEach-Object -Process {$_.LastWriteTime = Get-Date}

Didn't work for me, it looks like the problem lies in the cloud database on my end too

[frankhoff]

Apparently the 01.01.1970 is not the only problematic date, with the update to version 3.4.4 on Linux Mint 20.3 some files suddenly got the modified at 07.02.2106.

Fixing on the client side works with find . -newermt 2106-02-06 -ls and find . -newermt 2106-02-06 -exec touch -m {} on a Linux client, but the error seems to remain on the server or database for some reason.

[lars-sh]

We also had some cases of "Cannot sync due to invalid modification time", but instead of the other solutions, which work on the client-side, we solved this on the server-side, which might be a good idea in case you've no access to some clients yourself.

Solution

I found some entries in the data base table oc_filecache with either an unexpectedly low mtime or an unexpectedly high one (way larger than current timestamp). You can use the following query for that.

In a second step I replaced the mtime value using the storage_mtime value. That's it.

SELECT *
FROM oc_filecache
WHERE oc_filecache.mtime <> oc_filecache.storage_mtime
  AND oc_filecache.mtime NOT BETWEEN 1000000000 AND UNIX_TIMESTAMP()
LIMIT 100
;

[pwnorbitals]

I found some entries in the data base table oc_filecache with either an unexpectedly low mtime or an unexpectedly high one (way larger than current timestamp). You can use the following query for that.

In a second step I replaced the mtime value using the storage_mtime value. That's it.

Can confirm this fixed the issue on my end

[jreed1701]

We also had some cases of "Cannot sync due to invalid modification time", but instead of the other solutions, which work on the client-side, we solved this on the server-side, which might be a good idea in case you've no access to some clients yourself.

Solution

I found some entries in the data base table oc_filecache with either an unexpectedly low mtime or an unexpectedly high one (way larger than current timestamp). You can use the following query for that.

In a second step I replaced the mtime value using the storage_mtime value. That's it.

SELECT *
FROM oc_filecache
WHERE oc_filecache.mtime <> oc_filecache.storage_mtime
  AND oc_filecache.mtime NOT BETWEEN 1000000000 AND UNIX_TIMESTAMP()
LIMIT 100
;

I wanted to try this out but I'm not an SQL expert. I was able to get into my PSQL database, connect to the database with oc_filecache table, however, running the command caused issues.

ERROR: function unix_timestamps() does not exists.   

Is this SQL statement for mysql instead? If so, can someone help me undertand the error message and/or tweak it for PSQL?

[jreed1701]

I ended up using:

UPDATE oc_filecache SET mtime = storage_mtime WHERE mtime < 86400;

However, it didn't work for me because BOTH mtime and storage_mtime for those entries were 0.

I ended up getting the current unix timstamp manually, and then replacing the text "1234567890" with it.

UPDATE oc_filecache SET mtime = 1234567890 WHERE mtime < 86400;

[cramaboule]

@andyxheli Thanks andy. I tested these steps. find yournextcloudfilelocation/ ! -newermt 1970-01-02 > damaged.txt and similar commands did not return any files, same with the Nextcloud provided files.

There was only one folder affected: My main Documents folder, which had a date modified set to 2107 (86 years from now in the future). On the filesystem (server), the date was correct. On the client, the date was also correct.

⚠️ The wrong date was only in the nextcloud database. ⚠️

Here is how I solved this (just now).

cd /var/www/nextcloud/data/user/files/
touch -m Documents # Update only the last modified date to today
occ files:scan --path "user/files/Documents"' # scan again

tada:

What wasn't clear to me is that files:scan will only pick up files that changed.

Also updated nextcloud-client to 3.4.4 again and can confirm that the error disappeared.

This resolved my issue: find /path/to/data/directory/ -type f -newermt "1900-01-01" ! -newermt "1980-01-01" -exec touch {} +

I checked all files and dir up to 1980 THAN

find /path/to/data/directory/ -type f -newermt "2023-01-01" ! -newermt "2200-01-01" -exec touch {} +

From 2023 up to 2200 as I had files modified in futur (86 years from now) Even in files_trashbin !!! then: occ files:scan --path "user/files/Documents" --verbose or occ files:scan --all --verbose

You should not have any error

Now fixed !!!

Those files were on a Mac going to Windows by USB key and sync some years ago to my nexcloud server (Linux Nexcloud)

Hope this help !

C.

[lars-sh]

However, it didn't work for me because BOTH mtime and storage_mtime for those entries were 0.

@jreed1701, I just remember problems with the desktop client version 3.4.0, that might be related to your problem and that's further explained othere here: https://github.com/nextcloud/desktop/wiki/How-to-fix-the-error-invalid-or-negative-modification-date

[jreed1701]

However, it didn't work for me because BOTH mtime and storage_mtime for those entries were 0.

@jreed1701, I just remember problems with the desktop client version 3.4.0, that might be related to your problem and that's further explained othere here: https://github.com/nextcloud/desktop/wiki/How-to-fix-the-error-invalid-or-negative-modification-date

Thanks @lars-sh - I followed the procedure and running ./solvable_files.sh just returns and does nothing. I'm sure I have the right arguments because ./unsolvable_files.sh works just fine.

[vvvinny]

Thanks, this method works and I have fixed it. Here is the full command: find /path/to/data/directory/ -type f -name "*" -newermt 1970-01-01 ! -newermt 1970-01-02 -print0 | xargs -0 touch -m

Resolved with this command, but expanded date to 1969-12-31.

[jreed1701]

I got fed up, wiped my nextcloud server to start over. I ended up using the 3.3.6 client on Windows and that worked fine. I don't plan to upgrade until there's some definitive fix in the next release. Hopefully the admins communicate that clearly.

[benklop]

in my case i think this occurred when I changed the folders being synced in the client while a sync was trying to pull down a bunch of files. After checking an additional folder and pressing apply, I started receiving these errors and the mtime in the database doesn't seem correct any longer.

[shibco]

This bug burnt me severely in 2021 as mentioned in the OP, and its amazing to see this regression emerge again, first on Windows and now on Linux (Fedora 35). I literally have no real way of handling this with my users except to blanket ban the desktop clients, which is wild.

[vasyugan]

I see this even though there are no actual invalid modification times, neither on the server nor on the client.

[szelek]

How can I run solvable_files.sh when Nextcloud and DB are installed as separate docker images, and files are located on external NAS?

[Edukarl]

I see this even though there are no actual invalid modification times, neither on the server nor on the client.

I have the same problem. No invalid modification time on the files. Seems I just moved some files to a different folder on the client at the wrong time (during upload?).

EDIT: Turns out, the did have the invalid 01.01.1970 modification time. Just not on the server itself, but only on the client. So the have been uploaded but syncing to other clients did not work.

[skito]

I'm having the same problem on desktop client running on Win10. MacOS and mobile clients are running without issues.

Tried to fix the mtimes with server side commands suggested above and also do occ files:scan --all, but actually everything on the server was fine.

So I started reverting back old client versions one by one and the the first version to fix that problem was version 3.3.6... No issues there. This is definitely client side issue and I can't believe there've been 12 new releases since, that still didn't fix the issue including the latest 3.5.0

Nextcloud team please address this bug with more serious attention. It's obvious that a change between 3.3.6 and 3.4.0 which is further replicated in newer versions is causing this issue.

I'm ready to assist and provide log data if needed. Just let me know.

[TomMaSS]

Also have this bug on 3.5.0 client with sync dir what setup as auto loaded photos from mobile device

[raphmarot]

Fixed with

find /path/to/datadir/ -type f -mtime +36500 -exec touch {} \;

then

sudo -u www-data /path/to/php /path/to/nextcloud/occ files:scan --all

[jcastro]

I'm not able to fix this issue either, need to fix it on server side as client side doesn't have all the server files

[shibco]

A few days after installing a new copy of the Nextcloud desktop client and running solveable_files.sh, this issue is back!