immich icon indicating copy to clipboard operation
immich copied to clipboard

feat(server): optimize person thumbnail generation

Open mertalev opened this issue 1 year ago • 7 comments
trafficstars

Description

Person thumbnails are currently generated by cropping to a buffer and passing this buffer to sharp to resize. This is inefficient:

  1. It ends up generating two thumbnails instead of one (the buffer is implicitly a new JPEG file)
  2. libvips is optimized for longer image processing pipelines, so separating the workload here is worse for performance
  3. It lowers quality as person thumbnails are compressed at least three times: once due to the use of the compressed preview image, a second time with the buffer, and a third time with the resize

This PR combines cropping and resizing to the same pipeline for better performance and less compressed thumbnails. Additionally, it uses the original image to generate the thumbnail to further reduce compression artifacts, scaling the bounding box dimensions as needed.

How Has This Been Tested?

I ran facial recognition on all assets and confirmed the person thumbnails are accurately cropped and look normal.

Before:

person-thumbnail-before

After:

person-thumbnail-after

mertalev avatar Feb 29 '24 01:02 mertalev

Deploying immich with  Cloudflare Pages  Cloudflare Pages

Latest commit: 0309ad1
Status: ✅  Deploy successful!
Preview URL: https://e0654264.immich.pages.dev
Branch Preview URL: https://feat-optimize-thumbnail-gene.immich.pages.dev

View logs

I tested with a new instance, and the facial thumbnail crop is off image

I also got some errors due to bad dimensions.

image

alextran1502 avatar Feb 29 '24 03:02 alextran1502

Hmm that's weird, it was working well when I tested it. I'll do more testing tomorrow

mertalev avatar Feb 29 '24 04:02 mertalev

@alextran1502 Can you confirm if you still have the issue as of the latest commit? I'm wondering if you maybe tried it with an older commit

mertalev avatar Mar 06 '24 04:03 mertalev

@alextran1502 Can you confirm if you still have the issue as of the latest commit? I'm wondering if you maybe tried it with an older commit

I will take a look again and report back

alextran1502 avatar Mar 06 '24 05:03 alextran1502

I tried on a slice of my library and reproduced the issue. I think it's because of #3569.

mertalev avatar May 01 '24 00:05 mertalev

Can confirm that handling orientation fixed all the bad crops.

mertalev avatar May 01 '24 01:05 mertalev

@alextran1502 Could you test this PR when you get a chance? I want to be sure the cropping issues you noticed are fixed now.

mertalev avatar May 06 '24 00:05 mertalev

Great work! Thank you for the discovery and the fix!

alextran1502 avatar May 08 '24 13:05 alextran1502