Feature request: Enable multicore/multiprocess classification

[Tirpitz93]

Describe theFeature I run an older CPU without AVX/modern instructions, but I do have many cores. would be really nice to make better use of these cores by splitting up the classification run.

Desired behavior A clear and concise description of what you expected to happen.

Recognize (please complete the following information):

• JS-only mode: [e.g. Yes]
• Enabled modes: All

Server (please complete the following information):

• Nextcloud: 24
• OS: DEBIAN
• RAM: 64GB
• Processor Architecture x86 (Older Xeon)

[marcelklehr]

Hello @Tirpitz93

You should be able to manually run multiple classification runs in parallel by running one of these commands

php occ background-job:execute OCA\Recognize\BackgroundJobs\ClassifyFacesJob
php occ background-job:execute OCA\Recognize\BackgroundJobs\ClassifyImagenetJob # object recognition
php occ background-job:execute OCA\Recognize\BackgroundJobs\ClassifyLandmarksJob
php occ background-job:execute OCA\Recognize\BackgroundJobs\ClassifyMovinetJob # video recognition
php occ background-job:execute OCA\Recognize\BackgroundJobs\ClassifyMusicnnJob # music genre recognition

[marcelklehr]

Ah, but that command is only available in nextcloud 25

[Tirpitz93]

Could the process be batched up and split of several processes?

[marcelklehr]

Yes, that's the idea.

Running e.g. php occ background-job:execute OCA\Recognize\BackgroundJobs\ClassifyImagenetJob will classify in batches of 100 files. If you run many instances of this process you get parallel processing.

[marcelklehr]

Oh. the feature didn't even land in v25 yet :/

https://github.com/nextcloud/server/pull/30359

[reox]

I tried php occ background-job:execute OCA\Recognize\BackgroundJobs\ClassifyFacesJob on NC 25.0.0 but only get:

Job with ID 0 could not be found in the database

Is this related to the open PR or something different?

I would like to classify all images at once, because for a couple thousand images, I do not want to wait some days... (assuming the maximum is 400/hour, right?)

[marcelklehr]

Is this related to the open PR or something different?

Yes, this is related to the PR not being merged, yet.

assuming the maximum is 400/hour, right?

In v3.1.1 the max is now 1200 per hour.

[reox]

Yes, this is related to the PR not being merged, yet. ah okay, thanks for the confirmation!