Unable to upload large files, server 500 error when external storage uses S3 backend

[xinjiajuan]

Steps to reproduce

1.Connect the S3 storage backend using the external storage feature

Note: The S3 backend uses a local self-built minio server. The bucket has stored some files, but owncloud does not show it.

2.Uploading files about less than 500MB works fine. 3.After uploading a file larger than about 500MB, it prompts: 500 code error occurred when merging file chunks

Expected behaviour

See 3.NO in Steps to reproduce

Actual behaviour

See 3.NO in Steps to reproduce

Server configuration

Operating system: Ubuntu Server LTS 22.04

Web server: N/A

Database: N/A

PHP version: N/A

ownCloud version: ownCloud 10.10.0 (stable)

Where did you install ownCloud from: Docker/Image owncloud/server:latest

Signing status (ownCloud 9.0 and above): N/A

Login as admin user into your ownCloud and access 
http://example.com/index.php/settings/integrity/failed 
paste the results into https://gist.github.com/ and puth the link here.

The content of config/config.php: N/A

Log in to the web-UI with an administrator account and click on
'admin' -> 'Generate Config Report' -> 'Download ownCloud config report'
This report includes the config.php settings, the list of activated apps
and other details in a well sanitized form.

or 

If you have access to your command line run e.g.:
sudo -u www-data php occ config:list system
from within your ownCloud installation folder

*ATTENTION:* Do not post your config.php file in public as is. Please use one of the above
methods whenever possible. Both, the generated reports from the web-ui and from occ config:list
consistently remove sensitive data. You still may want to review the report before sending.
If done manually then it is critical for your own privacy to dilligently
remove *all* host names, passwords, usernames, salts and other credentials before posting.
You should assume that attackers find such information and will use them against your systems.

List of activated apps: N/A

If you have access to your command line run e.g.:
sudo -u www-data php occ app:list
from within your ownCloud installation folder.

Are you using external storage, if yes which one: S3

Are you using encryption: S3 Self yes

Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...

LDAP configuration (delete this part if not used)

With access to your command line run e.g.:
sudo -u www-data php occ ldap:show-config
from within your ownCloud installation folder

Without access to your command line download the data/owncloud.db to your local
computer or access your SQL server remotely and run the select query:
SELECT * FROM `oc_appconfig` WHERE `appid` = 'user_ldap';


Eventually replace sensitive data as the name/IP-address of your LDAP server or groups.

Client configuration

Browser: Chrome 103.0.5060.114

Operating system: Windows 11

Logs

Web server error log

192.168.2.95 - - [13/Jul/2022:14:09:35 +0000] "PUT /remote.php/dav/uploads/admin/web-file-upload-ad0b3ca80501b6585c474881d20cbbf2-1657721342592/2086666240 HTTP/1.1" 201 551 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36"
192.168.2.95 - - [13/Jul/2022:14:09:36 +0000] "GET /ocs/v2.php/apps/notifications/api/v1/notifications?format=json HTTP/1.1" 200 798 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36"
192.168.2.95 - - [13/Jul/2022:14:09:36 +0000] "PUT /remote.php/dav/uploads/admin/web-file-upload-ad0b3ca80501b6585c474881d20cbbf2-1657721342592/2097152000 HTTP/1.1" 201 553 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36"
192.168.2.95 - - [13/Jul/2022:14:09:36 +0000] "PUT /remote.php/dav/uploads/admin/web-file-upload-ad0b3ca80501b6585c474881d20cbbf2-1657721342592/2107637760 HTTP/1.1" 201 552 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36"
192.168.2.95 - - [13/Jul/2022:14:09:36 +0000] "PUT /remote.php/dav/uploads/admin/web-file-upload-ad0b3ca80501b6585c474881d20cbbf2-1657721342592/2118123520 HTTP/1.1" 201 552 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36"
192.168.2.95 - - [13/Jul/2022:14:09:36 +0000] "PUT /remote.php/dav/uploads/admin/web-file-upload-ad0b3ca80501b6585c474881d20cbbf2-1657721342592/2128609280 HTTP/1.1" 201 552 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36"
192.168.2.95 - - [13/Jul/2022:14:09:36 +0000] "PUT /remote.php/dav/uploads/admin/web-file-upload-ad0b3ca80501b6585c474881d20cbbf2-1657721342592/2139095040 HTTP/1.1" 201 552 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36"
192.168.2.95 - - [13/Jul/2022:14:09:36 +0000] "PUT /remote.php/dav/uploads/admin/web-file-upload-ad0b3ca80501b6585c474881d20cbbf2-1657721342592/2149580800 HTTP/1.1" 201 552 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36"
192.168.2.95 - - [13/Jul/2022:14:09:37 +0000] "GET /index.php/apps/files/ajax/getstoragestats.php?dir=%2F%E5%A4%87%E4%BB%BD HTTP/1.1" 200 876 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36"
192.168.2.95 - - [13/Jul/2022:14:10:06 +0000] "GET /ocs/v2.php/apps/notifications/api/v1/notifications?format=json HTTP/1.1" 200 835 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36"
[Wed Jul 13 14:10:22.048575 2022] [php7:error] [pid 505] [client 192.168.2.95:61663] PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 367001632 bytes) in /var/www/owncloud/apps/files_external/3rdparty/guzzlehttp/streams/src/Utils.php on line 66
192.168.2.95 - - [13/Jul/2022:14:09:36 +0000] "MOVE /remote.php/dav/uploads/admin/web-file-upload-ad0b3ca80501b6585c474881d20cbbf2-1657721342592/.file HTTP/1.1" 500 497 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36"
192.168.2.95 - - [13/Jul/2022:14:10:22 +0000] "GET /index.php/apps/files/ajax/getstoragestats.php?dir=%2F%E5%A4%87%E4%BB%BD HTTP/1.1" 200 876 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36"
192.168.2.95 - - [13/Jul/2022:14:10:36 +0000] "GET /ocs/v2.php/apps/notifications/api/v1/notifications?format=json HTTP/1.1" 200 835 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36"
192.168.2.95 - - [13/Jul/2022:14:11:06 +0000] "GET /ocs/v2.php/apps/notifications/api/v1/notifications?format=json HTTP/1.1" 200 835 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36"
192.168.2.95 - - [13/Jul/2022:14:11:36 +0000] "GET /ocs/v2.php/apps/notifications/api/v1/notifications?format=json HTTP/1.1" 200 835 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36"

ownCloud log (data/owncloud.log)

Browser log

Insert your browser log here, this could for example include:

a) The javascript console log
b) The network log 
c) ...

Reporter's opinion

mentioned in the log:

Allowed memory size of 536870912 bytes exhausted (tried to allocate 367001632 bytes) in /var/www/owncloud/apps/files_external/3rdparty/guzzlehttp/streams/src/Utils.php on line 66

error, but I'm not familiar with php programming and don't know how to adjust the memory footprint.