server [Bug]: Nextcloud does not see file change in S3 external storage unless occ files:scan

⚠️ This issue respects the following points: ⚠️

[x] This is a bug, not a question or a configuration/webserver/proxy issue.
[x] This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
[x] Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
[x] I agree to follow Nextcloud's Code of Conduct.

Bug description

Hi,

Issue: Nextcloud is supposed to follow any change in S3 directory on access, as configured:

However, when I add/delete a file on S3 side, Nextcloud does not update when I access the directory containing the file. Only if I do a manual scan occ files:scan or if I try to access the file in Nextcloud, which results in nothing, then after a browser refresh the list is correct.

Workaround: I added a file scan every 5 min in cron, but this may not be good for S3 server to be interrogated for nothing, and we have a 5min delay.

Steps to reproduce

deploy Nextcloud
deploy external storage app
configure S3 remote
add file in S3
see if the file appear in Nextcloud immediately => the file is not present

Expected behavior

Any access on Nextcloud will show correct info of file in remote.

Nextcloud Server version

30

Operating system

Other

PHP engine version

PHP 8.2

Web server

Nginx

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

Fresh Nextcloud Server install

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

[ ] Default user-backend (database)
[x] LDAP/ Active Directory
[ ] SSO - SAML
[ ] Other

Configuration report

{
    "system": {
        "htaccess.RewriteBase": "\/",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "debug": true,
        "check_data_directory_permissions": false,
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "forwarded_for_headers": [
            "HTTP_X_FORWARDED_FOR"
        ],
        "skeletondirectory": "\/dev\/null",
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "password": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "upgrade.disable-web": true,
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": "***REMOVED SENSITIVE VALUE***"
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "30.0.6.2",
        "overwrite.cli.url": "http:\/\/localhost",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "ldapProviderFactory": "OCA\\User_LDAP\\LDAPProviderFactory",
        "filelocking.enabled": false
    }
}

List of activated Apps

Enabled:
  - activity: 3.0.0
  - bruteforcesettings: 3.0.0
  - cloud_federation_api: 1.13.0
  - comments: 1.20.1
  - dav: 1.31.1
  - federatedfilesharing: 1.20.0
  - files: 2.2.0
  - files_downloadlimit: 3.0.0
  - files_external: 1.22.0
  - files_pdfviewer: 3.0.0
  - files_sharing: 1.22.0
  - logreader: 3.0.0
  - lookup_server_connector: 1.18.0
  - notifications: 3.0.0
  - oauth2: 1.18.1
  - password_policy: 2.0.0
  - provisioning_api: 1.20.0
  - serverinfo: 2.0.0
  - settings: 1.13.0
  - support: 2.0.0
  - survey_client: 2.0.0
  - systemtags: 1.20.0
  - text: 4.1.0
  - theming: 2.6.0
  - twofactor_backupcodes: 1.19.0
  - updatenotification: 1.20.0
  - user_ldap: 1.21.0
  - viewer: 3.0.0
  - workflowengine: 2.12.0
Enabled:
  - activity: 3.0.0
  - bruteforcesettings: 3.0.0
  - cloud_federation_api: 1.13.0
  - comments: 1.20.1
  - dav: 1.31.1
  - federatedfilesharing: 1.20.0
  - files: 2.2.0
  - files_downloadlimit: 3.0.0
  - files_external: 1.22.0
  - files_pdfviewer: 3.0.0
  - files_sharing: 1.22.0
  - logreader: 3.0.0
  - lookup_server_connector: 1.18.0
  - notifications: 3.0.0
  - oauth2: 1.18.1
  - password_policy: 2.0.0
  - provisioning_api: 1.20.0
  - serverinfo: 2.0.0
  - settings: 1.13.0
  - support: 2.0.0
  - survey_client: 2.0.0
  - systemtags: 1.20.0
  - text: 4.1.0
  - theming: 2.6.0
  - twofactor_backupcodes: 1.19.0
  - updatenotification: 1.20.0
  - user_ldap: 1.21.0
  - viewer: 3.0.0
  - workflowengine: 2.12.0Disabled:
  - admin_audit: 1.20.0
  - app_api: 4.0.6 (installed 4.0.6)
  - circles: 30.0.0 (installed 30.0.0)
  - contactsinteraction: 1.11.0 (installed 1.11.0)
  - dashboard: 7.10.0 (installed 7.10.0)
  - encryption: 2.18.0
  - federation: 1.20.0 (installed 1.20.0)
  - files_reminders: 1.3.0 (installed 1.3.0)
  - files_trashbin: 1.20.1 (installed 1.20.1)
  - files_versions: 1.23.0 (installed 1.23.0)
  - firstrunwizard: 3.0.0 (installed 3.0.0)
  - nextcloud_announcements: 2.0.0 (installed 2.0.0)
  - photos: 3.0.2 (installed 3.0.2)
  - privacy: 2.0.0 (installed 2.0.0)
  - recommendations: 3.0.0 (installed 3.0.0)
  - related_resources: 1.5.0 (installed 1.5.0)
  - sharebymail: 1.20.0 (installed 1.20.0)
  - suspicious_login: 8.0.0
  - twofactor_nextcloud_notification: 4.0.0
  - twofactor_totp: 12.0.0-dev
  - user_status: 1.10.0 (installed 1.10.0)
  - weather_status: 1.10.0 (installed 1.10.0)
  - webhook_listeners: 1.1.0-dev (installed 1.1.0-dev)

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

Additional info

No response

Jun 02 '25 08:06 antoinetran

Ok I found interesting info!!

When creating the mount point with occ, I have the issue. Then if I change in GUI the value of "Check for change" from "Once every direct access" (let's write it "OEDA") to "Never", and then the reverse, now the issue is gone.

When showing the mount points info with occ files_external:list, the column of mount options is the same, whether I configured with occ or with the GUI the "Check for change" to "OEDA":

+----------+-------------------------------------+-----------+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------+------------------+--------------------------+
| Mount ID | Mount Point                         | Storage   | Authentication Type | Configuration                                                                                                                                                                                                                                        | Options              | Applicable Users | Applicable Groups        |
+----------+-------------------------------------+-----------+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------+------------------+--------------------------+
| 6        | /REDACTED                           | Amazon S3 | Access key          | bucket: "***REMOVED SENSITIVE VALUE***", hostname: "REDACTED", port: "443", region: "", use_ssl: true, use_path_style: true, legacy_auth: false, useMultipartCopy: true, key: "***REMOVED SENSITIVE VALUE***", secret: "***REMOVED SENSITIVE VALUE***" | enable_sharing: true |                  | admin, REDACTED       |
+----------+-------------------------------------+-----------+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------+------------------+--------------------------+

If I set to "Never", then I get filesystem_check_changes: 0 on the mount option column, next to enable_sharing: true option.

This let me to believe that if I set it explicitely to 1 (OEDA), then the behavior is fixed, and yes, this is the case! Workaround:

occ files_external:option "${s3Id}" filesystem_check_changes 1

So! I believe if we do a occ files_external:create, it will create the mount point without setting the value of filesystem_check_changes. The GUI interprets the absence of this key as if its value is 1 (OEDA). But in reality, an absence of key amounts to value 0 (Never)

Jun 09 '25 17:06 antoinetran

I confirm my statements! Looking at mariadb database nextcloud

MariaDB [nextcloud]> select * from oc_external_options;
+-----------+----------+--------------------------+-------+
| option_id | mount_id | key                      | value |
+-----------+----------+--------------------------+-------+
|        15 |        4 | enable_sharing           | true  |
|        30 |        6 | enable_sharing           | true  |
|        32 |        6 | filesystem_check_changes | 1     |
+-----------+----------+--------------------------+-------+

Here you can see one mount_id = 4 automatically configured: filesystem_check_changes is missing and one mount_id = 6 automatically configured, with explicit value set to OEDA.

Jun 09 '25 17:06 antoinetran

So two possibles solutions for me:

the files_external:create automatically explicitely set the filesystem_check_changes to 1, consistent with the default option set in nextcloud code https://github.com/nextcloud/server/blob/v31.0.5/apps/files_external/lib/Command/ListCommand.php#L154
or the GUI interprets the absence of key as "Never" instead of "Once every direct access".

Of course the option 1 is the best, because it reproduces the same behavior when creating mount via GUI: the option is set to OEDA by default.

Jun 09 '25 17:06 antoinetran

server server copied to clipboard

[Bug]: Nextcloud does not see file change in S3 external storage unless occ files:scan

⚠️ This issue respects the following points: ⚠️

Bug description

Steps to reproduce

Expected behavior

Nextcloud Server version

Operating system

PHP engine version

Web server

Database engine version

Is this bug present after an update or on a fresh install?

Are you using the Nextcloud Server Encryption module?

What user-backends are you using?

Configuration report

List of activated Apps

Nextcloud Signing status

Nextcloud Logs

Additional info

server
server copied to clipboard