FTP file listing slow, Nextcloud makes new connection on each read

[zeigerpuppy]

Steps to reproduce

1. create an FTP (or FTPS) Share in External Storages settings
2. browse to FTP folders in Nextcloud
3. look at ProFTPd logs while browsing

Expected behaviour

Once an FTP connection is established, it should be maintained for the duration of the session. However, Nextcloud immediately closes the session after a directory listing. This results in delays due to the connecting needing to be re-established as the user browses folders.

When connecting from an external client (Filezilla), listings are much faster as the session is left open.

Actual behaviour

New connection established for each folder retrieved. The session is not left active. Disabling previews does not change the behaviour.

Server configuration

Operating system: Debian Jessie

Web server: Apache2

Database: MariaDB

PHP version: 7.0

Nextcloud version: 12.0.1

Updated from an older Nextcloud/ownCloud or fresh install: Updated from V.11 up

Where did you install Nextcloud from: main release files

Login as admin user into your Nextcloud and access 
http://example.com/index.php/settings/integrity/failed 
No errors have been found.

List of activated apps:

App list

Enabled:
  - activity: 2.5.2
  - admin_notifications: 1.0.0
  - announcementcenter: 3.1.0
  - apporder: 0.4.0
  - audioplayer: 2.0.2
  - bookmarks: 0.10.1
  - bruteforcesettings: 1.0.2
  - calendar: 1.5.3
  - comments: 1.2.0
  - contacts: 1.5.3
  - dav: 1.3.0
  - external: 2.0.3
  - federatedfilesharing: 1.2.0
  - federation: 1.2.0
  - files: 1.7.2
  - files_external: 1.3.0
  - files_markdown: 1.0.1
  - files_pdfviewer: 1.1.1
  - files_retention: 1.1.2
  - files_sharing: 1.4.0
  - files_texteditor: 2.4.1
  - files_trashbin: 1.2.0
  - files_versions: 1.5.0
  - files_videoplayer: 1.1.0
  - firstrunwizard: 2.1
  - gallery: 17.0.0
  - gpxpod: 2.2.0
  - impersonate: 1.0.1
  - keeweb: 0.4.0
  - logreader: 2.0.0
  - lookup_server_connector: 1.0.0
  - nextcloud_announcements: 1.1
  - notes: 2.3.1
  - notifications: 2.0.0
  - oauth2: 1.0.5
  - ocsms: 1.12.1
  - onlyoffice: 1.1.3
  - password_policy: 1.2.2
  - provisioning_api: 1.2.0
  - quota_warning: 1.0.1
  - rainloop: 5.0.0
  - serverinfo: 1.2.0
  - sharebymail: 1.2.0
  - spreed: 2.0.1
  - survey_client: 1.0.0
  - systemtags: 1.2.0
  - tasks: 0.9.5
  - theming: 1.3.0
  - twofactor_backupcodes: 1.1.1
  - updatenotification: 1.2.0
  - user_external: 0.4
  - workflowengine: 1.2.0
Disabled:
  - admin_audit
  - encryption
  - ojsxc
  - richdocuments
  - user_ldap

Nextcloud configuration:

Config report

{
    "system": {
        "instanceid": "oc0j4h8ro9en",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "nextcloud.myserver.net"
        ],
        "trusted_proxies": [
            "10.0.2.**"
        ],
        "datadirectory": "\/var\/www\/nextcloud\/data",
        "overwrite.cli.url": "http:\/\/nextcloud.myserver.net",
        "dbtype": "mysql",
        "version": "12.0.1.5",
        "dbname": "nextcloud",
        "dbhost": "database1.myserver",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "logtimezone": "UTC",
        "installed": true,
        "ldapIgnoreNamingRules": false,
        "ldapProviderFactory": "\\OCA\\User_LDAP\\LDAPProviderFactory",
        "loglevel": 3,
        "mail_smtpmode": "smtp",
        "mail_from_address": "mailer",
        "mail_domain": "myserver.net",
        "mail_smtpauthtype": "PLAIN",
        "mail_smtpauth": 1,
        "mail_smtphost": "mail.myserver.net",
        "mail_smtpport": "25",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "user_backends": [
            {
                "class": "OC_User_IMAP",
                "arguments": [
                    "{mail.myserver.net:143\/imap}"
                ]
            }
        ],
        "appstoreurl": "",
        "appstore.experimental.enabled": true,
        "maintenance": false,
        "theme": "",
        "updater.release.channel": "stable",
        "default_language": "en_GB"
    }
}

Are you using external storage, if yes which one: FTP

Are you using encryption: no

Are you using an external user-backend, if yes which one: IMAP

Client configuration

Browser: Firefox 55.0.3

Operating system: OS X Mavericks

Logs

Nextcloud log (data/nextcloud.log)

Nextcloud log

no relevant errors

[viisauksena]

we also have this issue, and while we have some TB of Data, and hundreds of nested folders with dozens of thousands files we actually have a initial sync run for 2 days now. We expect to take it some more days - which is somehow weird and not realy acceptable behaviour. in proftp logs we see this connect/close behaviour described. we do not really need searching or guessed sizes of whatever folder, or there should be other ways to run a index of an ftp.

[xtu]

I have 1032 folders in the FTP root directory. NC takes few minutes to list folders. In comparison, FileZilla list all the folders instantly.

[techsupport-noah]

This shouldn't be labeled as enhancement, since ftp is not working like this... labeling bug would be better since there wasnt any change since 2017...

[raphaelbernhart]

Any changes?