immich icon indicating copy to clipboard operation
immich copied to clipboard

fix(cli): Concurrency not fully using queue potential

Open Tiefseetauchner opened this issue 6 months ago • 9 comments

Hey!

Tl;dr the cli was chunking the concurrency which lead to a lot of needless wait time. With this change, the cli will correctly hash concurrency number of files at the same time.

Long story:

In asset.ts there was code handling the concurrency parameter but it was waiting for each file to complete hashing before tackling the next. The "solution" seems to have been to chunk files and process those concurrently in the queue. Which ever only has a single entry. So it's... Not reaaaaally a queue.

The solution is to throw all the items in the queue, wait for all of them to complete before moving on. Another improvement I made was to collect all items that were checked and only after they've been hashed send them to the server in one large mega request. Reducing chatter is my specialty.

I did the same thing to the upload queue.

Hope this is fine to just make a PR for ya'll without a prior issue!

Best, Lena <3

Tiefseetauchner avatar Aug 15 '24 18:08 Tiefseetauchner