docker icon indicating copy to clipboard operation
docker copied to clipboard
verified publisher icon nextcloud

WebDAV: Temporary high consumption of disk space when copying files

Open proninyaroslav opened this issue 3 years ago • 5 comments

How to use GitHub

  • Please use the 👍 reaction to show that you are affected by the same issue.
  • Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
  • Subscribe to receive notifications on status change and new comments.

I'm using an oficcial Nextcloud Docker image. Nginx reverse proxy is installed on the host system. The data volume is located on a separate HDD, and the app volume itself is in root on the SSD. I noticed this problem back in version 22.x: if I copy large files using WebDAV, the space on the root partition of the SSD runs out very quickly (/tmp of the host system uses tmpfs). At the same time, copying fails and in the log I see the following error, since the root disk space is full:

[PHP] Error: fread(): Write of 8192 bytes failed with errno=28 No space left on device at /var/www/html/apps/files_external/3rdparty/icewind/streams/src/Wrapper.php#55

PUT /cloud/remote.php/webdav/Videos/file.mkv

I tried to look for a similar problem on the forum and found what PHP caused the problem and the solution is to use a different path for sys_temp_dir. I tried to use the following php.ini:

sys_temp_dir = "/var/www/html/data/tmp"

/var/www/html/data is located on another HDD, so this should have solved the problem. I put this in my docker-compose.yml:

volumes:
  - ./php.ini:/usr/local/etc/php/conf.d/php_custom.ini

but that didn't solve the problem. I decided that it might be worth moving the entire /tmp directory of the image to tmpfs:

tmpfs:
  - /tmp

but that also gave nothing, as it turned out /tmp of the image is always empty, it looks like Nextcloud Docker doesn't use it.

So what is the reason for the high consumption of disk space? Where is this directory located in the Nextcloud image that it uses for temporary files when copying?

Server configuration

Operating system: Fedora 35 Server

Web server: Apache (Docker)

Database: MariaDB

PHP version: 8.0.14

Nextcloud version: Nextcloud Hub II (23.0.0)

Updated from an older Nextcloud/ownCloud or fresh install: no

Where did you install Nextcloud from: Docker

Signing status:

Signing status 
No errors have been found.

List of activated apps:

App list 
Enabled:
  - accessibility: 1.9.0
  - activity: 2.15.0
  - admin_audit: 1.13.0
  - bruteforcesettings: 2.3.0
  - calendar: 3.0.4
  - circles: 23.0.0
  - cloud_federation_api: 1.6.0
  - comments: 1.13.0
  - contacts: 4.0.7
  - contactsinteraction: 1.4.0
  - dashboard: 7.3.0
  - dav: 1.21.0
  - federatedfilesharing: 1.13.0
  - federation: 1.13.0
  - files: 1.18.0
  - files_accesscontrol: 1.13.0
  - files_external: 1.15.0
  - files_pdfviewer: 2.4.0
  - files_retention: 1.12.0
  - files_rightclick: 1.2.0
  - files_sharing: 1.15.0
  - files_trashbin: 1.13.0
  - files_versions: 1.16.0
  - files_videoplayer: 1.12.0
  - firstrunwizard: 2.12.0
  - logreader: 2.8.0
  - lookup_server_connector: 1.11.0
  - music: 1.5.0
  - news: 17.0.1
  - nextcloud_announcements: 1.12.0
  - notes: 4.3.0
  - notifications: 2.11.1
  - oauth2: 1.11.0
  - password_policy: 1.13.0
  - photos: 1.5.0
  - privacy: 1.7.0
  - provisioning_api: 1.13.0
  - recommendations: 1.2.0
  - richdocuments: 5.0.1
  - serverinfo: 1.13.0
  - settings: 1.5.0
  - sharebymail: 1.13.0
  - support: 1.6.0
  - survey_client: 1.11.0
  - systemtags: 1.13.0
  - tasks: 0.14.2
  - text: 3.4.0
  - theming: 1.14.0
  - twofactor_backupcodes: 1.12.0
  - twofactor_totp: 6.2.0
  - updatenotification: 1.13.0
  - user_ldap: 1.13.0
  - user_status: 1.3.1
  - viewer: 1.7.0
  - weather_status: 1.3.0
  - workflowengine: 2.5.0
Disabled:
  - encryption

Nextcloud configuration:

Config report 
{
    "system": {
        "htaccess.RewriteBase": "\/",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "password": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "localhost",
            "192.168.1.220",
            "proninyaroslav.ru"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "23.0.0.10",
        "overwriteprotocol": "https",
        "overwritehost": "proninyaroslav.ru",
        "overwritewebroot": "\/cloud",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "theme": "",
        "loglevel": 2,
        "maintenance": false,
        "app_install_overwrite": [
            "richdocumentscode"
        ],
        "ldapProviderFactory": "OCA\\User_LDAP\\LDAPProviderFactory",
        "overwrite.cli.url": "***REMOVED SENSITIVE VALUE***",
        "default_phone_region": "ru_RU"
    }
}

Are you using external storage, if yes which one: local

Are you using encryption: no

Are you using an external user-backend, if yes which one: Webdav

Client configuration

Browser: Firefox

Operating system: Fedora 35

Logs

Web server error log

Web server error log 
Insert your webserver log here

Nextcloud log (data/nextcloud.log)

Nextcloud log 
[PHP] Error: fread(): Write of 8192 bytes failed with errno=28 No space left on device at /var/www/html/apps/files_external/3rdparty/icewind/streams/src/Wrapper.php#55

PUT /cloud/remote.php/webdav/Videos/file.mkv

Browser log

Browser log 
Insert your browser log here, this could for example include:

a) The javascript console log
b) The network log
c) ...

proninyaroslav avatar Jan 18 '22 16:01 proninyaroslav

Don't even the developers know the answer to this question 🤔 ?

proninyaroslav avatar Feb 14 '22 08:02 proninyaroslav

Refer to https://github.com/nextcloud/server/issues/30282

There is apprarent a behavior change in the webdav client. Now all files over webdav are downloaded to the nextcloud /tmp first then served to the client.

It seems the nextcloud developer has no idea about this.

liejuntao001 avatar Feb 19 '22 21:02 liejuntao001

@liejuntao001 I tried to mount /tmp as tmpfs in the Docker (message above), but the volume of disk space still increased during uploading. Maybe I do something wrong?

proninyaroslav avatar Feb 20 '22 07:02 proninyaroslav

Any update on this?

I cannot copy any large files to my s3 instance anymore. Making enough space for large file is impossible without getting more storage (whether permanently or to move files to it temporarily)

kentarospin98 avatar Nov 12 '22 12:11 kentarospin98

@kentarospin98 There are no updates because the issue is incorrectly tagged as a question. It's most likely a bug though. @J0WI @szaimen @kangaroo72 Can any of you revisit this issue?

proninyaroslav avatar Nov 12 '22 14:11 proninyaroslav