Handle large text files that would exhaust php memory_limit

[fmkaiser]

[!NOTE]

• Currently exhausts PHP memory_limit
• Near term: Reject > X MB (e.g. memory_limit-128 or, safer, min(memory_limit-128, 100)
• Longer term: Implement streaming if deemed desirable

---

Large text files can exhaust the php memory limit when opened:

{"app":"PHP","method":"PUT","url":"/index.php/apps/text/session/9999578225/create","message":"Allowed memory size of 2147483648 bytes exhausted (tried to allocate 2900994904 bytes) at /var/www/nextcloud_32/lib/private/Files/Storage/Local.php#294"}

In the UI, this leads to a mostly white page with a generic error symbol at the bottom (screenshot below).

To Reproduce Steps to reproduce the behavior:

1. Generate & upload text file larger than php memory limit, for example with:
   dd if=/dev/urandom bs=1M count=2048 | base64 > large.txt
2. Open file in the text editor, either as a logged in user or via a public share link

Expected behavior Ideally - but probably difficult to implement -the file would be opened and displayed in chunks.

Alternatively, some configurable cutoff file size after which a clear error such as "file too large" is displayed to the user would be helpful.

For comparison, the onlyoffice editor has a configurable maxDownloadBytes parameter that defaults to 100M.

Screenshots

Server details:

• Nextcloud version: Nextcloud 31.0.8 EE, also reproduced with 32.0.0 CE.
• PHP Version: 8.2-fpm
• Database: MariaDB 10.11

Client details:

• Chrome 141 @ OpenSUSE Leap 16 (and others)

Logs

Nextcloud log (data/nextcloud.log)

{"app":"PHP","method":"PUT","url":"/index.php/apps/text/session/9999578225/create","message":"Allowed memory size of 2147483648 bytes exhausted (tried to allocate 2900994904 bytes) at /var/www/nextcloud_32/lib/private/Files/Storage/Local.php#294"}

Browser console

Failed to load resource: the server responded with a status of 500 ()

[joshtrichards]

Indeed. Thanks for the report!

Short-term: Regardless, we should probably clamp this at the active PHP memory_limit (minus a reasonable margin) to avoid this kind of brokenness.

Longer term: The solution comes down to whether supporting large text files (e.g., >~100 MiB) is even desirable enough to justify the implementation effort.

Out of curiosity, did you run into this in a real-world use case? If so, were you planning to actually edit the document, or just view it?

References:

• https://github.com/nextcloud/text/blob/109d3c3d482ac2cc5a0c131a16bd8ea67b492bae/lib/TextFile.php#L44-L47
• https://github.com/nextcloud/server/blob/5cb16af66e02fad13d0cd0f527517d329af02943/lib/private/Files/Storage/Local.php#L293-L294

[fmkaiser]

@joshtrichards We do see the issue in the logs of our productive instances:

Some of our users are working with and sharing large text files with genomic data, where a single file can be >100GB. I don't think they would neccessarily need or expect to be able to open them in the web editor however.

Most likely, people just click on the item particularily on public link shares by default, expecting to download the file.

So for our purpose, files larger than a certain size not being opened in the web editor would be perfectly fine.

[juliusknorr]

Let's tackle implementing the limit first to report back an error and block opening, bigger files need more general discussion anyways as the browser and editor rendering would likely also choke on such big files.