openvsx icon indicating copy to clipboard operation
openvsx copied to clipboard

Use InputStream instead of byte[] for file handling

Open amvanbaren opened this issue 2 years ago • 2 comments

Extension packages can be up to 512 MB and files within an extension package can be up to 32 MB. Luckily most extensions are a couple MB or less. However redhat.java, IBM.zopeneditor or ASF.apache-netbeans-java are extensions that are over a 100 MB in size.

To make the server more reliable, I'd suggest to store binary content in a temp file and reading from that file in chunks instead of directly loading all of the content in a byte array.

I think it is also better to permanently store the contents of a FileResource on the local file system instead of as a byte array in the database. This only applies when no cloud storage provider has been configured.

amvanbaren avatar Oct 13 '22 04:10 amvanbaren

Extra TODO: delete temp file (also in existing code, e.g. publish endpoint) after the server is done using it. https://github.com/eclipse/openvsx/pull/586#discussion_r1036826240

amvanbaren avatar Dec 06 '22 10:12 amvanbaren


Mdnou avatar May 26 '23 05:05 Mdnou