openvsx
openvsx copied to clipboard
Use InputStream instead of byte[] for file handling
Extension packages can be up to 512 MB and files within an extension package can be up to 32 MB.
Luckily most extensions are a couple MB or less. However redhat.java
, IBM.zopeneditor
or ASF.apache-netbeans-java
are extensions that are over a 100 MB in size.
To make the server more reliable, I'd suggest to store binary content in a temp file and reading from that file in chunks instead of directly loading all of the content in a byte array.
I think it is also better to permanently store the contents of a FileResource
on the local file system instead of as a byte array in the database. This only applies when no cloud storage provider has been configured.
Extra TODO: delete temp file (also in existing code, e.g. publish endpoint) after the server is done using it. https://github.com/eclipse/openvsx/pull/586#discussion_r1036826240