facerecognition icon indicating copy to clipboard operation
facerecognition copied to clipboard

Multiple external model instances

Open 36grad opened this issue 2 months ago • 4 comments

Summary: with this change multiple instances of the external model can be used to analyze multiple images in parallel.

Motivation: Since the recent change that analysis now uses more than one core the external model is kind of pointless (because it's only analyzing one image in one thread / using one core). That is especially sad if you have (for whatever reasons) a machine with a lot of cores and RAM that could easily analyze more than one image at one.

Enter "Multiple external model instances": With this change you can use multiple external model instances in parallel. For that, the new ImageProcessingWithMultipleExternalModelInstancesTask has been added which brings together the code from the original ImageProcessingTask and the ExternalModel and whips that up with curl_multi_exec to asynchronically send image analysis requests to the external model instances.

Configuration: All relevant settings on the facerecognition app side can be set using the occ face:setup command. Usage is displayed with occ face:setup -h.

The by far easiest solution is to use docker compose for the external model. There are only two changes to the docker-compose.yml required:

  1. you need to map a port range to the containers internal port rather than a single port mapping. The number of ports must be at least the number of instances: The following configuration line will use port 8083 through 8090.
     ports:
       - "8083-8090:5000"
  1. you specify that replicas shall be deployed: The following configuration line will result in a total of 8 instances of the external model.
     deploy:
       mode: replicated
       replicas: 8

Each replica will listen on subsequent ports, starting with the first port in above configured port range.

Changes to facerecognition: I tried to make as little chanegs to the original code as possible. Three things were unavoidable:

  1. Add the related system settings to SettingsService.
  2. Change the FaceRecognitionBackgroundTask to add a cleanup method in order to be able to react to timeout.
  3. Introduce a new class for the Task and schedule it in the BackgroundService.

Besides that, I modified the SetupCommand to have access to more settings and to show the current settings a bit nicer :-) Also I have added Nextcloud's internal logger to the FaceRecognitionContext so that we can send messages to the Nextcloud log file.

Final words: I have tried this on my server and on a machine with 16GB memory I can run 8 instances in parallel (with a temp image size of 1600x1200).

36grad avatar Apr 21 '24 19:04 36grad