Less Cache writes during Uploads (right now every 8 Kbytes)

[DiegoPino]

trafficstars

Is your feature request related to a problem? Please describe.

During a file Upload operation, files are copied using a constant/8 Kbytes chunks.

As seen here: https://github.com/ankitpokhrel/tus-php/blob/a466a9b835f3ea29ec4df7e37a2dc9e00d194304/src/File.php#L331

On every offset/bytes increment the Cache is Set using the new Offset value, all this inside a While driven by a seek. This is a blocking operation and the While eventually (at the end of the seek or if something goes wrong) will exist and return the offset. Writing to cache on every 8Kbytes seems excessive and could be just done before returning the offset?

Describe the solution you'd like Move the $this->cache->set($key, ['offset' => $this->offset]); after the while exists.

Describe alternatives you've considered I might be missing some edge case/parallel operation? of why the cache needs to be written very 8 Kbytes? At least, on a DB based cache implementation, this effectively delays the actual File availability (the client is done sending) considerably.

Additional context

Any pointers on why this behavior exists would be greatly appreciated. I locally moved the set cache outside of the while and my performance on very large files improved dramatically. Also, thanks a lot for your excellent code.

[irmantas]

I believe that with resumable uploads the system should automatically know where to resume. However, an 8 KB chunk size is quite small - it results in 128 cache writes per 1MB. Because the chunk size is currently defined as constant, it might be better to make it configurable.

[DiegoPino]

@irmantas this chunk here is not the TuS chunk. the Tus Chunk is managed by the client (JS) normally. This part of the code here is how pieces already uploaded are copied via PHP to its final destination file (when assembling). And the problem is the excessive updates on the cache.