nextcloud
`occ files:scan -all` Makes all folders show that they were just changed
Steps to reproduce
- Setup Nextcloud 10
- Connect to external SMB folders
- Use occ files:scan -all to scan for changes
Expected behaviour
New files or folders should be shown but no changes to their last update time and date unless it did change.
Actual behaviour
All folders on SMB shares show that they were updated or modified when the scan took place.
Server configuration
Ubuntu 16.4
Web server: Apache
Database: MySQL
PHP version: 7.0.8
Nextcloud version: (see Nextcloud admin page) 10
Updated from an older Nextcloud/ownCloud or fresh install: Fresh install
Where did you install Nextcloud from: Downloaded from Nextcloud.com
Signing status:
Signing status
Login as admin user into your Nextcloud and access
http://example.com/index.php/settings/integrity/failed
paste the results here.
List of activated apps:
App list
If you have access to your command line run e.g.:
sudo -u www-data php occ app:list
from within your Nextcloud installation folder
The content of config/config.php:
Config report
If you have access to your command line run e.g.:
sudo -u www-data php occ config:list system
from within your Nextcloud installation folder
or
Insert your config.php content here
(Without the database password, passwordsalt and secret)
Are you using external storage, if yes which one: local/smb/sftp/... SMB Are you using encryption: yes/no No Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/... No
LDAP configuration (delete this part if not used)
LDAP config
With access to your command line run e.g.:
sudo -u www-data php occ ldap:show-config
from within your Nextcloud installation folder
Without access to your command line download the data/owncloud.db to your local
computer or access your SQL server remotely and run the select query:
SELECT * FROM `oc_appconfig` WHERE `appid` = 'user_ldap';
Eventually replace sensitive data as the name/IP-address of your LDAP server or groups.
Browser:
Operating system:
Logs
Web server error log
Web server error log
Insert your webserver log here
Nextcloud log
Insert your Nextcloud log here
Browser log
Insert your browser log here, this could for example include:
a) The javascript console log
b) The network log
c) ...
It's the only way I have been able to get Nextcloud to pick up or recognize new files placed on the smb server. With only a few files it works without but after several gigs of files have been accessed and hundreds of folders, It just doesn't seem to look any further. I do have Check for Changes set to Once every direct access. But does that mean once every access via a sync client? Because that is the access most of my clients are using. If the check for changes should be working then I believe that is also listed as a bug that I found yesterday.
Same problem. 37 GiG of data, the only way ist to use "occ files:scan -all" Version 10.0.2
But does that mean once every access via a sync client? Because that is the access most of my clients are using.
I think it does yes. But let's confirm by @icewind1991
The sync client only accesses the users home folder and will thus only find top level changes.
With SMB, you can run an occ command that actively listen for smb changes.
This issue will be fixed with https://github.com/nextcloud/server/pull/3449
I scanned (occ files:scan -all) new local folders with 12.0.4 and there modification date is not correctly reported.
Well as per it's description: will rescan all files of all known users
Maybe --unscanned helps preventing that?
I dropped oc_filecache and launched occ files:scan -all --unscanned. All directory are 1s hold. I don't get why directory are managed differently than files on this.
You should not drop oc_filecache, it's not a cache but an index. The naming can't be changed easily which is why we are still using it.
Yes thanks, I know that not a cache. I should have tested by creating a new directory. I got excited.
Had this issue with nc 12 too. I saw that there was a storage_mtime field with the correct values in the oc_filecache table.
Just execute this sql statement after each occ files:scan :
UPDATE oc_filecache SET mtime=storage_mtime
or from the shell :
mysql -u root -p -e "UPDATE oc_filecache SET mtime=storage_mtime" nextcloud
Had this issue with nc 12 too. I saw that there was a storage_mtime field with the correct values in the oc_filecache table.
Just execute this sql statement after each occ files:scan :
UPDATE oc_filecache SET mtime=storage_mtimeor from the shell :
mysql -u root -p -e "UPDATE oc_filecache SET mtime=storage_mtime" nextcloud
Briliant, works like a charm as a workaround :ok_hand: Thanks!
What would be the postgresql command that does the same?
Edit:
su - postgres
postgres@example ~ % psql
postgres=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+----------+----------+---------+---------+-----------------------
nextcloud | something | UTF8 | C.UTF-8 | C.UTF-8 | something=CTc/something
[snip]
(4 rows)
postgres=# \connect nextcloud
You are now connected to database "nextcloud" as user "postgres".
nextcloud=# UPDATE oc_filecache SET mtime=storage_mtime;
UPDATE 38594
I just had this issue with Nextcloud 23.0.2, Ubuntu 20.04, MariaDB 10.3.32, apache2. Solution mentioned by @libreGit works. Are there any logs I can provide? Edit: Server error log is literally empty. Nextcloud log also didn't have anything new after the scan --all command.
Hi, please update to at least 23.0.12 and report back if it fixes the issue. Thank you!
Hi, please update to at least 23.0.12 and report back if it fixes the issue. Thank you!
Did you mean 23.0.11 ? I don't see a 23.0.12 release in Changelog (nor in update for that matter).
I too confirm the bug with the latest 25.0.1.1 version. While migrating the data between two virtual machines, I have used:
scp -rp source destination
to preserve the modification times. The folder stats are correct.
However, after triggering the scan, all folders appear to have been last modified at the moment of the copy. Values in the storage_mtime column are correct, but the mtime defaulted to what seem to be the time of the operation.
Thank you all for this very useful thread.
I also see this behaviour in Nextcloud 25.0.0 RC3
When a folder is added to Nexcloud either by dropping it in to the WebUI OR rsync'ing it in to the datadir and then running occ files:scan --all that folder is given a current Timestamp and not the os-level modified TS of the folder.
However, under normal usage perhaps that behavior makes sense? It's only a problem when you are setting up a new NC instance with your OLD data and for the first sync only you want to keep all the timestamps from the "old" NC instance.
So I wonder if the fix here is not to change the existing code but to add a new parameter to occ files:scan -all?
e.g: occ files:scan --all --preserve_os_mod_ts that tells it to use the folders os-level modified TS in the mtime and storage_mtime columns of the NC oc_filecache table. And then just using occ files:scan --all could continue to work as it currently does?
Just my two cents!
Flex
