immich icon indicating copy to clipboard operation
immich copied to clipboard

[BUG] Upload sames images multiples times create duplicates

Open Nonobis opened this issue 3 years ago • 11 comments

Describe the bug

On web, when uploading same images multiple times, the interface display multiples times the pictures. I don't know if image are physycally duplicated, on only on display.

Task List

  • [X] I have read thoroughly the README setup and installation instructions.
  • [X] If my setup is different, I have included my docker-compose file.
  • [ ] I have included my redacted .env file.
  • [X] I have included information on my machine, and environment.

To Reproduce Steps to reproduce the behavior:

  1. Go to Web
  2. Click on Upload
  3. Upload x pictures
  4. Wait ...
  5. Return to 2. and re-upload same images

Expected behavior No duplicate images ....

Screenshots image image

Desktop (please complete the following information):

  • OS : Unraid
  • Latest docker image

Nonobis avatar Jun 22 '22 18:06 Nonobis

When you refresh the page, is the duplicated image still there?

alextran1502 avatar Jun 22 '22 18:06 alextran1502

yes, tested 1 or 2 hour after ...and duplicate are still here

Nonobis avatar Jun 22 '22 21:06 Nonobis

Hmm interesting, so that means the image is duplicated physically. So before uploading the file to the server, the web will check for this format

web-{file_name}-{last_modified_date_in_utc}

If the file is modified or has different names, then it will upload.

Can you help me check the file? Modified can be seen as moving a file from one location to another

alextran1502 avatar Jun 23 '22 01:06 alextran1502

From one father to another, happy late father's day!

alextran1502 avatar Jun 23 '22 01:06 alextran1502

i think i have understand the problem... i have made a first upload of this pictures. The date/time of file was wrong, so i made a quick program for extract OrginalDatetime from the metadata and re-apply it to files before re-uploading (as there is no deletion for now).

It was my assumption your where calculating an hash of file for testing if picture was truly unique. (i use this nuget in .Net CoenM.ImageSharp.ImageHash, for calculating hash in a personal program using image....).

PS : For french translation, i will make a pr when #246 will be merged :)

Nonobis avatar Jun 23 '22 04:06 Nonobis

i have found that duplicate image have double extension in web ui *.jpg.jpg i don't known if it can help....

Nonobis avatar Jun 23 '22 04:06 Nonobis

image image

Nonobis avatar Jun 23 '22 04:06 Nonobis

Regarding the jpg.jpg extension: It seems to me that images uploaded from the web create a double .jpg extensions and the images uploaded from the app creates a single

SirBogner avatar Jun 23 '22 06:06 SirBogner

i think i have understand the problem... i have made a first upload of this pictures. The date/time of file was wrong, so i made a quick program for extract OrginalDatetime from the metadata and re-apply it to files before re-uploading (as there is no deletion for now).

It was my assumption your where calculating an hash of file for testing if picture was truly unique. (i use this nuget in .Net CoenM.ImageSharp.ImageHash, for calculating hash in a personal program using image....).

PS : For french translation, i will make a pr when #246 will be merged :)

This can be a solution, related to the deduplication on the server.

alextran1502 avatar Jun 23 '22 12:06 alextran1502

Regarding the jpg.jpg extension: It seems to me that images uploaded from the web create a double .jpg extensions and the images uploaded from the app creates a single

@SirBogner This might be a bug. Do you have a way to reproduce? If so, would you mind open an issue? I will take a look at it

alextran1502 avatar Jun 23 '22 12:06 alextran1502

When i have the webview open and upload images via iOS, i see duplicates in the web view. When I download them, the files have different IDs. After a page refresh i don't see duplicates any longer.

V 1.13.0

m4rkv5 avatar Jun 25 '22 10:06 m4rkv5