previewgenerator icon indicating copy to clipboard operation
previewgenerator copied to clipboard

Multithreading Support

Open cmdrkarl opened this issue 1 year ago • 3 comments

Hello,

can multithreading be added to use modern CPUs?

Thanks, Kurt

cmdrkarl avatar Dec 02 '23 22:12 cmdrkarl

This would be really nice, first time generating previews on an instance with a decent amount of media is pain, it took on my instance the about 24h on a i7 3770k..

All that is needed is to start multiple preview generations simultaneously set by a command line option such that some cores can be left for serving nextcloud.. I'm not sure how it works when executed trough php but ffmpeg supports multithreading per default, so one could keep a single queue but generate each image a lot faster instead..

This would also help when a lot of images are being uploaded at the same time, as currently its possible on my instance to upload a lot faster than the preview generation keeps up..

MrRinkana avatar Feb 01 '24 19:02 MrRinkana

I suspect this will be left as h2non/imaginary is being recommended instead of this app - and in that setup multithreading will be used!

MrRinkana avatar Feb 23 '24 11:02 MrRinkana

What would happen if you ran this command:

bash -c 'for i in {1..4}; do (occ preview:pre-generate -vvv &); done'

Is this a hack? Or, would it destroy my system? From what I gather, it would spawn 4 different instances of pre-generate at the same time... is this okay/allowed?

Update

So I just went ahead and tested with one folder. It spawned 4 instances and they all scanned the same file... not what we want... But, my library is organized by Year and Month, so I changed it to:

sudo htop
bash -c 'for i in {1..4}; do (occ preview:pre-generate -vvv --path="/username/files/Photos/2024/0$i" &); done'

Because of how the second command is run, you cannot input the sudo password, so you have to sudo something first before running the command... I choose sudo htop. What the second command did is that it spawned four processes looking in my Photos directory, year 2024, then the $i corresponds to what month I want... Not "multithreading" but does make scanning four times faster in this example...

iamdoubz avatar Apr 04 '24 16:04 iamdoubz