immich
immich copied to clipboard
immich-app
Upload error: duplicate key value violates unique constraint "UQ_assets_owner_checksum"
The bug
This is a bug/error log that has been present for quite some time.
At least some of the checksummed images are confirmed to not be Live photos on the source device. All errors seem to come from one user, despite the instance having two users.
All images were taken on and uploaded from the iOS client. I suspect it's some kind of upload race condition. It seems like the error only appears a max of one time per file.
The OS that Immich Server is running on
Debian
Version of Immich Server
v1.123.0
Version of Immich Mobile App
v1.123.0
Platform with the issue
- [X] Server
- [ ] Web
- [X] Mobile
Your docker-compose.yml content
N/A
External Postgres
Your .env content
N/A
Reproduction steps
Upload an image
Relevant log output
2024-12-30 12:21:13.887 EST [3173250] immich@immich ERROR: duplicate key value violates unique constraint "UQ_assets_owner_checksum"
2024-12-30 12:21:13.887 EST [3173250] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \x0f9a2bdb7971e7d8bb390ce718f0dd6263f07597) already exists.
2024-12-30 12:21:13.887 EST [3173250] immich@immich STATEMENT: INSERT INTO "assets"("id", "deviceAssetId", "ownerId", "libraryId", "deviceId", "type", "status", "originalPath", "thumbhash"
, "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "fileCreatedAt", "localDateTime", "fileModifiedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "checksum", "duration
", "isVisible", "livePhotoVideoId", "originalFileName", "sidecarPath", "stackId", "duplicateId") VALUES (DEFAULT, $1, $2, $3, $4, $5, DEFAULT, $6, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAUL
T, $7, $8, $9, $10, $11, DEFAULT, DEFAULT, $12, $13, $14, DEFAULT, $15, DEFAULT, DEFAULT, DEFAULT) RETURNING "id", "status", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "isFa
vorite", "isArchived", "isExternal", "isOffline", "isVisible"
2024-12-30 12:21:17.670 EST [3173247] immich@immich ERROR: duplicate key value violates unique constraint "UQ_assets_owner_checksum"
2024-12-30 12:21:17.670 EST [3173247] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \x110514e48f93991b9499d6b2c1f8cd56d82c5298) already exists.
2024-12-30 12:21:17.670 EST [3173247] immich@immich STATEMENT: INSERT INTO "assets"("id", "deviceAssetId", "ownerId", "libraryId", "deviceId", "type", "status", "originalPath", "thumbhash"
, "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "fileCreatedAt", "localDateTime", "fileModifiedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "checksum", "duration
", "isVisible", "livePhotoVideoId", "originalFileName", "sidecarPath", "stackId", "duplicateId") VALUES (DEFAULT, $1, $2, $3, $4, $5, DEFAULT, $6, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAUL
T, $7, $8, $9, $10, $11, DEFAULT, DEFAULT, $12, $13, $14, DEFAULT, $15, DEFAULT, DEFAULT, DEFAULT) RETURNING "id", "status", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "isFa
vorite", "isArchived", "isExternal", "isOffline", "isVisible"
2024-12-30 12:21:18.517 EST [3173247] immich@immich ERROR: duplicate key value violates unique constraint "UQ_assets_owner_checksum"
2024-12-30 12:21:18.517 EST [3173247] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \x91d246c5904b88187e2effa7a166bdbecf169088) already exists.
2024-12-30 12:21:18.517 EST [3173247] immich@immich STATEMENT: INSERT INTO "assets"("id", "deviceAssetId", "ownerId", "libraryId", "deviceId", "type", "status", "originalPath", "thumbhash"
, "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "fileCreatedAt", "localDateTime", "fileModifiedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "checksum", "duration
", "isVisible", "livePhotoVideoId", "originalFileName", "sidecarPath", "stackId", "duplicateId") VALUES (DEFAULT, $1, $2, $3, $4, $5, DEFAULT, $6, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAUL
T, $7, $8, $9, $10, $11, DEFAULT, DEFAULT, $12, $13, $14, DEFAULT, $15, DEFAULT, DEFAULT, DEFAULT) RETURNING "id", "status", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "isFa
vorite", "isArchived", "isExternal", "isOffline", "isVisible"
2024-12-30 12:21:25.940 EST [3173247] immich@immich ERROR: duplicate key value violates unique constraint "UQ_assets_owner_checksum"
2024-12-30 12:21:25.940 EST [3173247] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \x554231ea0e5350be99b426f0904ab2bbd0972dc1) already exists.
2024-12-30 12:21:25.940 EST [3173247] immich@immich STATEMENT: INSERT INTO "assets"("id", "deviceAssetId", "ownerId", "libraryId", "deviceId", "type", "status", "originalPath", "thumbhash"
, "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "fileCreatedAt", "localDateTime", "fileModifiedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "checksum", "duration
", "isVisible", "livePhotoVideoId", "originalFileName", "sidecarPath", "stackId", "duplicateId") VALUES (DEFAULT, $1, $2, $3, $4, $5, DEFAULT, $6, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAUL
T, $7, $8, $9, $10, $11, DEFAULT, DEFAULT, $12, $13, $14, DEFAULT, $15, DEFAULT, DEFAULT, DEFAULT) RETURNING "id", "status", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "isFa
vorite", "isArchived", "isExternal", "isOffline", "isVisible"
2024-12-30 12:21:26.614 EST [3173247] immich@immich ERROR: duplicate key value violates unique constraint "UQ_assets_owner_checksum"
2024-12-30 12:21:26.614 EST [3173247] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \x33f32a52dd750289326d8c517f67c428644ac92e) already exists.
2024-12-30 12:21:26.614 EST [3173247] immich@immich STATEMENT: INSERT INTO "assets"("id", "deviceAssetId", "ownerId", "libraryId", "deviceId", "type", "status", "originalPath", "thumbhash", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "fileCreatedAt", "localDateTime", "fileModifiedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "checksum", "duration", "isVisible", "livePhotoVideoId", "originalFileName", "sidecarPath", "stackId", "duplicateId") VALUES (DEFAULT, $1, $2, $3, $4, $5, DEFAULT, $6, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, $7, $8, $9, $10, $11, DEFAULT, DEFAULT, $12, $13, $14, DEFAULT, $15, DEFAULT, DEFAULT, DEFAULT) RETURNING "id", "status", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "isVisible"
2024-12-30 12:21:32.670 EST [3173247] immich@immich ERROR: duplicate key value violates unique constraint "UQ_assets_owner_checksum"
2024-12-30 12:21:32.670 EST [3173247] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \x110514e48f93991b9499d6b2c1f8cd56d82c5298) already exists.
2024-12-30 12:21:32.670 EST [3173247] immich@immich STATEMENT: INSERT INTO "assets"("id", "deviceAssetId", "ownerId", "libraryId", "deviceId", "type", "status", "originalPath", "thumbhash", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "fileCreatedAt", "localDateTime", "fileModifiedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "checksum", "duration", "isVisible", "livePhotoVideoId", "originalFileName", "sidecarPath", "stackId", "duplicateId") VALUES (DEFAULT, $1, $2, $3, $4, $5, DEFAULT, $6, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, $7, $8, $9, $10, $11, DEFAULT, DEFAULT, $12, $13, $14, DEFAULT, $15, DEFAULT, DEFAULT, DEFAULT) RETURNING "id", "status", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "isVisible"
2024-12-29 04:40:29.631 EST [286228] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \xfc83718c4f963cd4c5768fff7060395210215eb1) already exists.
2024-12-30 12:21:13.887 EST [3173250] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \x0f9a2bdb7971e7d8bb390ce718f0dd6263f07597) already exists.
2024-12-30 12:21:17.670 EST [3173247] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \x110514e48f93991b9499d6b2c1f8cd56d82c5298) already exists.
2024-12-30 12:21:18.517 EST [3173247] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \x91d246c5904b88187e2effa7a166bdbecf169088) already exists.
2024-12-30 12:21:25.940 EST [3173247] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \x554231ea0e5350be99b426f0904ab2bbd0972dc1) already exists.
2024-12-30 12:21:26.614 EST [3173247] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \x33f32a52dd750289326d8c517f67c428644ac92e) already exists.
2024-12-30 12:21:32.670 EST [3173247] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \x110514e48f93991b9499d6b2c1f8cd56d82c5298) already exists.
2024-12-30 12:21:33.085 EST [3173247] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \x33f32a52dd750289326d8c517f67c428644ac92e) already exists.
2024-12-30 12:21:33.609 EST [3173247] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \x7f8b2d0f7b121850a7d7b5ca0cf6590cb4cc5b78) already exists.
2024-12-30 12:21:33.707 EST [3173247] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \x554231ea0e5350be99b426f0904ab2bbd0972dc1) already exists.
2024-12-30 12:21:34.295 EST [3173247] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \x58d5c391c848c75bd923735908100755b3456276) already exists.
2024-12-30 12:21:34.334 EST [3173247] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \x91d246c5904b88187e2effa7a166bdbecf169088) already exists.
2024-12-30 12:21:34.775 EST [3173247] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \xdb191ad14caee582a79e104d5d55ef82201caccf) already exists.
2024-12-30 12:21:34.832 EST [3173247] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \x0f9a2bdb7971e7d8bb390ce718f0dd6263f07597) already exists.
2024-12-30 12:21:34.992 EST [3173247] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \xdbf54a03f65f772fbc987827786bc32684a7f855) already exists.
2024-12-30 12:21:35.001 EST [3173247] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \x70fd9a5103f28bd175c4939541c9e1689dad3aeb) already exists.
2024-12-30 12:21:35.906 EST [3173247] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \x1346d64b035da8736bbbc0f07a0cbe0b21ce272d) already exists.
2024-12-30 12:21:36.298 EST [3173247] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \x45b673caadc3f3e33d32aa1b6fe35ce354fdd48b) already exists.
2024-12-30 12:21:36.626 EST [3173247] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \x2de02c3e3debf675b6199c66b25f650e73585754) already exists.
2024-12-30 12:21:37.405 EST [3173247] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \xec7e8e038669c1e3060f6f62b15b5e626d2af124) already exists.
2024-12-30 12:21:39.741 EST [3173247] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \x3feb81bc06fa5bd7585d417a5a299c213aa91cd4) already exists.
2024-12-30 12:21:40.277 EST [3173247] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \x3fea3f61fafd36c2c66a645b2c5d0eb35cc0745a) already exists.
2024-12-30 12:21:40.879 EST [3173250] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \x2d10b6716ed7086972b40e60be450309f3ac40e6) already exists.
2024-12-30 12:21:41.526 EST [3173250] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \x575d2cbd9b86bc30d241a87e06ebd1250686c320) already exists.
2024-12-30 12:21:41.802 EST [3173250] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \xf08ea1600cfaeccfd3837f6b2158b1ce7292c0ad) already exists.
2025-01-02 13:04:53.611 EST [118038] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \xb40e611d175b59bbf1d3b945e136c902ff891a28) already exists.
2025-01-02 13:04:53.707 EST [118038] immich@immich DETAIL: Key ("ownerId", checksum)=(c36ed9e6-26c9-4b89-b457-5b078897482d, \x18dfc4c838c25c7b9379ad29d7f143edb2497610) already exists.
### Additional information
_No response_
version: 1.123.0 OS: Debian 11(bullseye) SoC: RK3588
[Nest] 17 - 01/03/2025, 3:27:33 PM LOG [Api:EventRepository] Websocket Connect: mKYXKKV-X9yflSPuAABF
[Nest] 7 - 01/03/2025, 3:27:39 PM LOG [Microservices:LibraryService] Refreshing library bee24505-7455-44f3-b949-b617c326eafb for new assets
[Nest] 7 - 01/03/2025, 3:27:39 PM LOG [Microservices:LibraryService] Scanning library bee24505-7455-44f3-b949-b617c326eafb for removed assets
[Nest] 7 - 01/03/2025, 3:27:39 PM LOG [Microservices:LibraryService] Finished queueing check of 38 assets for library bee24505-7455-44f3-b949-b617c326eafb
[Nest] 7 - 01/03/2025, 3:27:41 PM ERROR [Microservices:MetadataService] Failed to extract live photo /mnt/photos2/U1833403067267301378/我的相册/Samsung_Galaxy_S8+.jpg: QueryFailedError: duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
QueryFailedError: duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async InsertQueryBuilder.execute (/usr/src/app/node_modules/typeorm/query-builder/InsertQueryBuilder.js:106:33)
at async SubjectExecutor.executeInsertOperations (/usr/src/app/node_modules/typeorm/persistence/SubjectExecutor.js:260:42)
at async SubjectExecutor.execute (/usr/src/app/node_modules/typeorm/persistence/SubjectExecutor.js:92:9)
at async EntityPersistExecutor.execute (/usr/src/app/node_modules/typeorm/persistence/EntityPersistExecutor.js:140:21)
at async MetadataService.applyMotionPhotos (/usr/src/app/dist/services/metadata.service.js:339:31)
at async MetadataService.handleMetadataExtraction (/usr/src/app/dist/services/metadata.service.js:164:9)
at async JobService.onJobStart (/usr/src/app/dist/services/job.service.js:148:28)
at async EventRepository.onEvent (/usr/src/app/dist/repositories/event.repository.js:134:13)
[Nest] 7 - 01/03/2025, 3:27:48 PM LOG [Microservices:TrashService] Queued 1 assets for deletion from the trash
[Nest] 7 - 01/03/2025, 3:27:48 PM WARN [Microservices:StorageService] Unable to remove file from disk
[Nest] 7 - 01/03/2025, 3:27:48 PM WARN [Microservices:StorageService] Error: EROFS: read-only file system, unlink '/mnt/photos2/U1833403067267301378/我的相册/Samsung_Galaxy_S8+.jpg'
[Nest] 17 - 01/03/2025, 3:27:54 PM WARN [Api:PersonService~9u6foc25] Unknown keys found: {
"machineLearning": {
"url": "http://immich-machine-learning:3003"
}
}
Sometimes got "Duplicate SOI marker"
[Nest] 7 - 01/03/2025, 2:50:17 PM LOG [Microservices:LibraryService] Importing new library asset: /mnt/photos2/U1833403067267301378/我的相册/Google_Pixel_2.jpg
[Nest] 7 - 01/03/2025, 2:50:17 PM LOG [Microservices:LibraryService] Importing new library asset: /mnt/photos2/U1833403067267301378/我的相册/Samsung_Galaxy_Z_Fold4.heic
[Nest] 7 - 01/03/2025, 2:50:17 PM LOG [Microservices:LibraryService] Importing new library asset: /mnt/photos2/U1833403067267301378/我的备份/来自： iPhone 12 Pro Max 的备份/相册备份/安卓 motion photo/Samsung_Galaxy_S8+.jpg
[Nest] 7 - 01/03/2025, 2:50:17 PM LOG [Microservices:LibraryService] Importing new library asset: /mnt/photos2/U1833403067267301378/我的相册/Google_Pixel_2.jpg
[Nest] 7 - 01/03/2025, 2:50:17 PM LOG [Microservices:LibraryService] Importing new library asset: /mnt/photos2/U1833403067267301378/我的相册/Samsung_Galaxy_Z_Fold4.heic
[Nest] 7 - 01/03/2025, 2:50:17 PM ERROR [Microservices:JobService] Unable to run job handler (library/library-sync-file): QueryFailedError: duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
[Nest] 7 - 01/03/2025, 2:50:17 PM ERROR [Microservices:JobService] QueryFailedError: duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async InsertQueryBuilder.execute (/usr/src/app/node_modules/typeorm/query-builder/InsertQueryBuilder.js:106:33)
at async SubjectExecutor.executeInsertOperations (/usr/src/app/node_modules/typeorm/persistence/SubjectExecutor.js:260:42)
at async SubjectExecutor.execute (/usr/src/app/node_modules/typeorm/persistence/SubjectExecutor.js:92:9)
at async EntityPersistExecutor.execute (/usr/src/app/node_modules/typeorm/persistence/EntityPersistExecutor.js:140:21)
at async LibraryService.handleSyncFile (/usr/src/app/dist/services/library.service.js:341:17)
at async JobService.onJobStart (/usr/src/app/dist/services/job.service.js:148:28)
at async EventRepository.onEvent (/usr/src/app/dist/repositories/event.repository.js:134:13)
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
[Nest] 7 - 01/03/2025, 2:50:17 PM ERROR [Microservices:JobService] Object:
{
"id": "bee24505-7455-44f3-b949-b617c326eafb",
"assetPath": "/mnt/photos2/U1833403067267301378/我的相册/Google_Pixel_2.jpg",
"ownerId": "2af2fdc1-f374-417e-9e0f-3660d26b9793"
}
[Nest] 7 - 01/03/2025, 2:50:17 PM ERROR [Microservices:JobService] Unable to run job handler (library/library-sync-file): QueryFailedError: duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
[Nest] 7 - 01/03/2025, 2:50:17 PM ERROR [Microservices:JobService] QueryFailedError: duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async InsertQueryBuilder.execute (/usr/src/app/node_modules/typeorm/query-builder/InsertQueryBuilder.js:106:33)
at async SubjectExecutor.executeInsertOperations (/usr/src/app/node_modules/typeorm/persistence/SubjectExecutor.js:260:42)
at async SubjectExecutor.execute (/usr/src/app/node_modules/typeorm/persistence/SubjectExecutor.js:92:9)
at async EntityPersistExecutor.execute (/usr/src/app/node_modules/typeorm/persistence/EntityPersistExecutor.js:140:21)
at async LibraryService.handleSyncFile (/usr/src/app/dist/services/library.service.js:341:17)
at async JobService.onJobStart (/usr/src/app/dist/services/job.service.js:148:28)
at async EventRepository.onEvent (/usr/src/app/dist/repositories/event.repository.js:134:13)
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
[Nest] 7 - 01/03/2025, 2:50:17 PM ERROR [Microservices:JobService] Object:
{
"id": "bee24505-7455-44f3-b949-b617c326eafb",
"assetPath": "/mnt/photos2/U1833403067267301378/我的相册/Samsung_Galaxy_Z_Fold4.heic",
"ownerId": "2af2fdc1-f374-417e-9e0f-3660d26b9793"
}
[Nest] 7 - 01/03/2025, 2:50:18 PM LOG [Microservices:MetadataService] Wrote motion photo video to upload/encoded-video/2af2fdc1-f374-417e-9e0f-3660d26b9793/14/0b/140b2f6e-744a-47c2-83eb-5c0a792beb80-MP.mp4
[Nest] 7 - 01/03/2025, 2:50:18 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-thumbnails): Error: VipsJpeg: ./lib/jpegli/decode_scan.cc:535: Incomplete scan detected.
VipsJpeg: ./lib/jpegli/decode_marker.cc:448: Duplicate SOI marker
[Nest] 7 - 01/03/2025, 2:50:18 PM ERROR [Microservices:JobService] Error: VipsJpeg: ./lib/jpegli/decode_scan.cc:535: Incomplete scan detected.
VipsJpeg: ./lib/jpegli/decode_marker.cc:448: Duplicate SOI marker
at Sharp.toBuffer (/usr/src/app/node_modules/sharp/lib/output.js:163:17)
at MediaRepository.decodeImage (/usr/src/app/dist/repositories/media.repository.js:57:68)
at MediaService.generateImageThumbnails (/usr/src/app/dist/services/media.service.js:161:63)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async MediaService.handleGenerateThumbnails (/usr/src/app/dist/services/media.service.js:111:25)
at async JobService.onJobStart (/usr/src/app/dist/services/job.service.js:148:28)
at async EventRepository.onEvent (/usr/src/app/dist/repositories/event.repository.js:134:13)
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 01/03/2025, 2:50:18 PM ERROR [Microservices:JobService] Object:
{
"id": "2a6b6254-163a-4670-a240-f93526241240",
"source": "upload"
}
I get this too, a bunch of times too fro Pixel motion pictures.
[Nest] 7 - 01/11/2025, 5:50:56 PM ERROR [Microservices:MetadataService] Failed to extract live photo /mnt/share-pictures/Android Gallery/Camera/PXL_20230408_222810785.MP.jpg: QueryFailedError: duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
QueryFailedError: duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async InsertQueryBuilder.execute (/usr/src/app/node_modules/typeorm/query-builder/InsertQueryBuilder.js:106:33)
at async SubjectExecutor.executeInsertOperations (/usr/src/app/node_modules/typeorm/persistence/SubjectExecutor.js:260:42)
at async SubjectExecutor.execute (/usr/src/app/node_modules/typeorm/persistence/SubjectExecutor.js:92:9)
at async EntityPersistExecutor.execute (/usr/src/app/node_modules/typeorm/persistence/EntityPersistExecutor.js:140:21)
at async MetadataService.applyMotionPhotos (/usr/src/app/dist/services/metadata.service.js:339:31)
at async MetadataService.handleMetadataExtraction (/usr/src/app/dist/services/metadata.service.js:164:9)
at async JobService.onJobStart (/usr/src/app/dist/services/job.service.js:148:28)
at async EventRepository.onEvent (/usr/src/app/dist/repositories/event.repository.js:134:13)
Perhaps related, I get [ERR_OUT_OF_RANGE] for some.
[Nest] 7 - 01/11/2025, 6:17:14 PM ERROR [Microservices:MetadataService] Failed to extract live photo /mnt/share-pictures/Database/2021/PXL_20210725_220218680.MP.jpg: RangeError [ERR_OUT_OF_RANGE]: The value of "position" is out of range. It must be >= -1 && <= 9007199254740991. Received -3589549
RangeError [ERR_OUT_OF_RANGE]: The value of "position" is out of range. It must be >= -1 && <= 9007199254740991. Received -3589549
at read (node:internal/fs/promises:692:5)
at fsCall (node:internal/fs/promises:465:18)
at FileHandle.read (node:internal/fs/promises:196:12)
at StorageRepository.readFile (/usr/src/app/dist/repositories/storage.repository.js:85:43)
at async MetadataService.applyMotionPhotos (/usr/src/app/dist/services/metadata.service.js:317:25)
at async MetadataService.handleMetadataExtraction (/usr/src/app/dist/services/metadata.service.js:164:9)
at async JobService.onJobStart (/usr/src/app/dist/services/job.service.js:148:28)
at async EventRepository.onEvent (/usr/src/app/dist/repositories/event.repository.js:134:13)
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
v1.124.2, docker.
Edit: Both errors occur reliably even if I manually rescan the metadata of the affected image one-by-one.
I encountered the same issue:
duplicate key value violates unique constraint "UQ_assets_owner_checksum"
Upon inspecting the row in the database, I found that it was a picture marked with a "trashed" status.
After permanently deleting the picture through the web UI, I was able to successfully upload it again from my iPhone.
Hope this helps!
I encoutered same issue.
Found why:
- Backed up photos from phone.
- Logged in via web Was trying to upload same videos(i downloaded them before from GPhotos), so instead of showing "Duplicate" it just showed error in web. Got same logs as above.
I encounter the same issue, but with a different entry path -- simply backing up from my Android Phone.
Instead of the file being marked properly as "already backed up", the Android App reads the server result as "Not backed up". This creates what looks like dupe entries on the Android App [The Already backedup from server read, the not backed-up one].
May or may not be due to this server side error (I do strongly suspect it is though), filed #16101 to track separately if needed.
I encountered the same issue:
duplicate key value violates unique constraint "UQ_assets_owner_checksum"Upon inspecting the row in the database, I found that it was a picture marked with a "trashed" status.
After permanently deleting the picture through the web UI, I was able to successfully upload it again from my iPhone.
Hope this helps!
Hm, this does not work for me. I see 2 pictures in Immich in iOS but I only have 1 photo stored locally. One was uploaded, second fails to upload as it is already a duplicate. I found over 100 photos that have this issue. I could go through each picture manually but that does not seem like a right thing to do. Can't really find a reliable way how to find these duplicate photos. To me, it seems like there was a bug in iOS 18.0 as photos taken before are fine. On 18.1, so far it seems okay
EDIT: I have tried to remove this particular photo but even when I did and reuploaded it in Immich app, the same issue reappeared. (I have emptied my trash) On web, the photo is deleted but immich app shows the thumbnail but when I open the photo, the screen is blank with an icon that the photo no longer exists (which is correct). Nothing happens when I trash the photo but when I reupload the original photo, this duplicate is again restored.
Database:
id ; deviceAssetId ; ownerId ; deviceId ; type ; originalPath ; fileCreatedAt ; fileModifiedAt ; isFavorite ; duration ; encodedVideoPath ; checksum ; isVisible ; livePhotoVideoId ; updatedAt ; createdAt ; isArchived ; originalFileName ; sidecarPath ; thumbhash ; isOffline ; libraryId ; isExternal ; deletedAt ; localDateTime ; stackId ; duplicateId ; status
661db72f-333a-4e6a-a5fe-c6236f2037d5 ; 1B08D236-0E27-4C1F-AEB6-999B9211BB18/L0/001 ; 15b0e8be-5295-47a5-8e93-b5e3cc15f295 ; a9f77229f356cc28f567acd279585cd59c85800a852672f569cfb5efdc00e304 ; VIDEO ; upload/upload/15b0e8be-5295-47a5-8e93-b5e3cc15f295/f4/b0/f4b022d1-d417-4917-a816-66c71086c008.mov ; 2024-11-05 08:30:57+00 ; 2024-11-06 19:05:17+00 ; f ; 00:00:02.005 ; upload/encoded-video/15b0e8be-5295-47a5-8e93-b5e3cc15f295/66/1d/661db72f-333a-4e6a-a5fe-c6236f2037d5.mp4 ; \x9b938853ea9f0ab6206a72453872e520b4c8bcf2 ; f ; ; 2025-02-15 16:45:07.642301+00 ; 2025-02-15 16:45:02.565432+00 ; f ; IMG_6466.mov ; ; ; f ; ; f ; ; 2024-11-05 09:30:57+00 ; ; ; active
1131bb97-6524-4bc4-83a1-9d50deb92db6 ; 1B08D236-0E27-4C1F-AEB6-999B9211BB18/L0/001 ; 15b0e8be-5295-47a5-8e93-b5e3cc15f295 ; a9f77229f356cc28f567acd279585cd59c85800a852672f569cfb5efdc00e304 ; IMAGE ; upload/upload/15b0e8be-5295-47a5-8e93-b5e3cc15f295/00/77/0077854d-c3af-4b88-8772-d312326fef49.HEIC ; 2024-11-05 08:30:58.052+00 ; 2024-11-06 19:05:17+00 ; f ; ; ; \xe9c996642381c20b9c94087866e999a121a7d8cf ; t ; 661db72f-333a-4e6a-a5fe-c6236f2037d5 ; 2025-02-15 16:45:03.251601+00 ; 2024-11-10 10:28:47.714492+00 ; f ; IMG_6466.HEIC ; ; \x62180e15029b9a809889778867777777a66f14fb45 ; f ; ; f ; ; 2024-11-05 09:30:58.052+00 ; ; ; active
Postgres logs:
Query failed : {
durationMs: 25.28949399999692,
error: PostgresError: duplicate key value violates unique constraint "UQ_assets_owner_checksum"
at ErrorResponse (/usr/src/app/node_modules/postgres/cjs/src/connection.js:788:26)
at handle (/usr/src/app/node_modules/postgres/cjs/src/connection.js:474:6)
at Socket.data (/usr/src/app/node_modules/postgres/cjs/src/connection.js:315:9)
at Socket.emit (node:events:524:28)
at addChunk (node:internal/streams/readable:561:12)
at readableAddChunkPushByteMode (node:internal/streams/readable:512:3)
at Readable.push (node:internal/streams/readable:392:5)
at TCP.onStreamRead (node:internal/stream_base_commons:189:23)
at TCP.callbackTrampoline (node:internal/async_hooks:130:17) {
severity_local: 'ERROR',
severity: 'ERROR',
code: '23505',
detail: 'Key ("ownerId", checksum)=(15b0e8be-5295-47a5-8e93-b5e3cc15f295, \\x9b938853ea9f0ab6206a72453872e520b4c8bcf2) already exists.',
schema_name: 'public',
table_name: 'assets',
constraint_name: 'UQ_assets_owner_checksum',
file: 'nbtinsert.c',
line: '663',
routine: '_bt_check_unique'
},
sql: 'insert into "assets" ("ownerId", "libraryId", "checksum", "originalPath", "deviceAssetId", "deviceId", "fileCreatedAt", "fileModifiedAt", "localDateTime", "type", "isFavorite", "isArchived", "duration", "isVisible", "originalFileName") values ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15) returning *',
params: [
'15b0e8be-5295-47a5-8e93-b5e3cc15f295',
,
<Buffer 9b 93 88 53 ea 9f 0a b6 20 6a 72 45 38 72 e5 20 b4 c8 bc f2>,
'upload/upload/15b0e8be-5295-47a5-8e93-b5e3cc15f295/12/65/1265fb85-6b90-42f6-8b13-06f025a9cc8b.mov',
'1B08D236-0E27-4C1F-AEB6-999B9211BB18/L0/001',
'a9f77229f356cc28f567acd279585cd59c85800a852672f569cfb5efdc00e304',
2024-11-05T08:30:58.000Z,
2024-11-06T19:05:17.000Z,
2024-11-05T08:30:58.000Z,
'VIDEO',
false,
false,
'0:00:00.000000',
,
'IMG_6466.mov'
]
}
[Nest] 16 - 02/15/2025, 6:25:25 PM DEBUG [Api:LoggingInterceptor~ido6ggaq] POST /api/assets 200 2324.99ms ::ffff:192.168.1.120
deviceAssetId=1B08D236-0E27-4C1F-AEB6-999B9211BB18/L0/001 deviceId=a9f77229f356cc28f567acd279585cd59c85800a852672f569cfb5efdc00e304 fileCreatedAt=2024-11-05T08:30:58.000Z fileModifiedAt=2024-11-06T19:05:17.000Z isFavorite=false duration=0:00:00.000000
[Nest] 16 - 02/15/2025, 6:25:25 PM VERBOSE [Api:LoggingInterceptor~ido6ggaq]
Query failed : {
durationMs: 8.13733699999284,
error: PostgresError: duplicate key value violates unique constraint "UQ_assets_owner_checksum"
at ErrorResponse (/usr/src/app/node_modules/postgres/cjs/src/connection.js:788:26)
at handle (/usr/src/app/node_modules/postgres/cjs/src/connection.js:474:6)
at Socket.data (/usr/src/app/node_modules/postgres/cjs/src/connection.js:315:9)
at Socket.emit (node:events:524:28)
at addChunk (node:internal/streams/readable:561:12)
at readableAddChunkPushByteMode (node:internal/streams/readable:512:3)
at Readable.push (node:internal/streams/readable:392:5)
at TCP.onStreamRead (node:internal/stream_base_commons:189:23)
at TCP.callbackTrampoline (node:internal/async_hooks:130:17) {
severity_local: 'ERROR',
severity: 'ERROR',
code: '23505',
detail: 'Key ("ownerId", checksum)=(15b0e8be-5295-47a5-8e93-b5e3cc15f295, \\xe9c996642381c20b9c94087866e999a121a7d8cf) already exists.',
schema_name: 'public',
table_name: 'assets',
constraint_name: 'UQ_assets_owner_checksum',
file: 'nbtinsert.c',
line: '663',
routine: '_bt_check_unique'
},
sql: 'insert into "assets" ("ownerId", "libraryId", "checksum", "originalPath", "deviceAssetId", "deviceId", "fileCreatedAt", "fileModifiedAt", "localDateTime", "type", "isFavorite", "isArchived", "duration", "isVisible", "livePhotoVideoId", "originalFileName") values ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16) returning *',
params: [
'15b0e8be-5295-47a5-8e93-b5e3cc15f295',
,
<Buffer e9 c9 96 64 23 81 c2 0b 9c 94 08 78 66 e9 99 a1 21 a7 d8 cf>,
'upload/upload/15b0e8be-5295-47a5-8e93-b5e3cc15f295/89/eb/89eb4825-5751-4c9c-9b29-142737583622.HEIC',
'1B08D236-0E27-4C1F-AEB6-999B9211BB18/L0/001',
'a9f77229f356cc28f567acd279585cd59c85800a852672f569cfb5efdc00e304',
2024-11-05T08:30:58.000Z,
2024-11-06T19:05:17.000Z,
2024-11-05T08:30:58.000Z,
'IMAGE',
false,
false,
'0:00:00.000000',
,
'661db72f-333a-4e6a-a5fe-c6236f2037d5',
'IMG_6466.HEIC'
]
}
I'm seeing this too. I have two Android phones, both logged into my Immich server as the same user, in case that is relevant.
It seems the server will log this error when you attempt to upload a duplicate without sending the x-immich-checksum header (which bypasses trying to save the upload if one already exists with the same checksum). It handles this condition, deleting the uploaded file and returning the id of the existing image with the matching checksum.
See here:
https://github.com/immich-app/immich/blob/a96bba4b26a4cec2f60f1680011e40265fa04f5b/server/src/services/asset-media.service.ts#L297
May want to add logging to indicate this error has been handled, and look at the clients performing the upload to see if they are using the x-immich-checksum header (it seems immich-go doesn't - that's where I saw this error).
Thanks for your research. I’m experiencing this with the use of the iOS app exclusively. Potentially background vs foreground issue? Are we sure it’s set in both those cases?
In my case, theese are not duplicates but rather ghost pictures, cannot find them by date or filename in the ios gallery nor immich photos.
Short background history:
- created immich server docker instance
- syncing constantly stopped by server issue (core dumps)
- increased resources of server
- syncing completed, a few showed pending which were the ghosts
- immich app was not in a consistent state, showing pictures not uploaded or not on device, therefore wiped and reinstalled app
- syncing showed only these ghosts at backup remaining assets which constantly failed with the same (as this issue) error
Getting this as well. immich-go. puling media from multiple sources and most likey a ton of dupes.
}
Query failed : {
durationMs: 0.968752000015229,
error: PostgresError: duplicate key value violates unique constraint "UQ_assets_owner_checksum"
at ErrorResponse (/app/immich/server/node_modules/postgres/cjs/src/connection.js:788:26)
at handle (/app/immich/server/node_modules/postgres/cjs/src/connection.js:474:6)
at Socket.data (/app/immich/server/node_modules/postgres/cjs/src/connection.js:315:9)
at Socket.emit (node:events:518:28)
at addChunk (node:internal/streams/readable:561:12)
at readableAddChunkPushByteMode (node:internal/streams/readable:512:3)
at Readable.push (node:internal/streams/readable:392:5)
at TCP.onStreamRead (node:internal/stream_base_commons:191:23)
at TCP.callbackTrampoline (node:internal/async_hooks:130:17) {
severity_local: 'ERROR',
severity: 'ERROR',
code: '23505',
detail: 'Key ("ownerId", checksum)=(64f36505-bbb6-416f-a818-a57751ceb707, \\x7e693d9577270f2342fb931a79f5245fa761fe7a) already exists.',
schema_name: 'public',
table_name: 'assets',
constraint_name: 'UQ_assets_owner_checksum',
file: 'nbtinsert.c',
line: '663',
routine: '_bt_check_unique'
},
sql: 'insert into "assets" ("ownerId", "libraryId", "checksum", "originalPath", "deviceAssetId", "deviceId", "fileCreatedAt", "fileModifiedAt", "localDateTime", "type", "isFavorite", "isArchived", "duration", "isVisible", "originalFileName") values ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15) returning *',
params: [
'64f36505-bbb6-416f-a818-a57751ceb707',
null,
<Buffer 7e 69 3d 95 77 27 0f 23 42 fb 93 1a 79 f5 24 5f a7 61 fe 7a>,
'/photos/upload/64f36505-bbb6-416f-a818-a57751ceb707/34/e2/34e216b6-ae07-4977-a101-2e99d43dde35.JPG',
'2023-11-18_115434 (2).JPG-5005342',
'DESKTOP-CHLB4HU',
2023-11-18T11:54:32.000Z,
2023-11-18T11:54:34.000Z,
2023-11-18T11:54:32.000Z,
'IMAGE',
false,
false,
'00:00:00.000000',
true,
'2023-11-18_115434 (2).JPG'
]
}
Latest immich-go version (0.25.0) avoids uploading duplicates better, so should see this a lot less.
Getting this as well. immich-go. puling media from multiple sources and most likey a ton of dupes.
I’m encountering the same issue when trying to upload my entire photo library (~20 years of photos) using immich-cli.
Many assets fail to upload with duplicate key value violates unique constraint "UQ_assets_owner_checksum" errors, which seem to happen consistently across large batches of older files.
The error details are very similar to those already described in this issue.
Let me know if there's anything I can provide (logs, asset metadata, etc.) to help reproduce or diagnose this at scale.
Query failed : {
durationMs: 3.6291810013353825,
error: PostgresError: duplicate key value violates unique constraint "UQ_assets_owner_checksum"
at ErrorResponse (/usr/src/app/node_modules/postgres/cjs/src/connection.js:790:26)
at handle (/usr/src/app/node_modules/postgres/cjs/src/connection.js:476:6)
at Socket.data (/usr/src/app/node_modules/postgres/cjs/src/connection.js:315:9)
at Socket.emit (node:events:518:28)
at addChunk (node:internal/streams/readable:561:12)
at readableAddChunkPushByteMode (node:internal/streams/readable:512:3)
at Readable.push (node:internal/streams/readable:392:5)
at TCP.onStreamRead (node:internal/stream_base_commons:189:23)
at TCP.callbackTrampoline (node:internal/async_hooks:130:17) {
severity_local: 'ERROR',
severity: 'ERROR',
code: '23505',
detail: 'Key ("ownerId", checksum)=(4a396719-b723-46b7-9573-980ab3b3f114, \\x78f05a470349ff727d87f8cfd97d7abe829c8967) already exists.',
schema_name: 'public',
table_name: 'assets',
constraint_name: 'UQ_assets_owner_checksum',
file: 'nbtinsert.c',
line: '663',
routine: '_bt_check_unique'
},
sql: 'insert into "assets" ("ownerId", "libraryId", "checksum", "originalPath", "deviceAssetId", "deviceId", "fileCreatedAt", "fileModifiedAt", "localDateTime", "type", "isFavorite", "isArchived", "duration", "isVisible", "originalFileName") values ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15) returning *',
params: [
'4a396719-b723-46b7-9573-980ab3b3f114',
null,
<Buffer 78 f0 5a 47 03 49 ff 72 7d 87 f8 cf d9 7d 7a be 82 9c 89 67>,
'upload/upload/4a396719-b723-46b7-9573-980ab3b3f114/2b/c1/2bc15ebb-38ea-4b08-99cf-aed586da48d1.jpg',
'2009-01-07_-14-57_CanonDIGITALIXUS850IS.jpg-2313635',
'CLI',
2013-02-09T16:33:48.904Z,
2013-02-09T16:33:48.904Z,
2013-02-09T16:33:48.904Z,
'IMAGE',
false,
false,
null,
true,
'2009-01-07_-14-57_Canon DIGITAL IXUS 850 IS.jpg'
]
}
Query failed : {
durationMs: 5.366627000272274,
error: PostgresError: duplicate key value violates unique constraint "UQ_assets_owner_checksum"
at ErrorResponse (/usr/src/app/node_modules/postgres/cjs/src/connection.js:790:26)
at handle (/usr/src/app/node_modules/postgres/cjs/src/connection.js:476:6)
at Socket.data (/usr/src/app/node_modules/postgres/cjs/src/connection.js:315:9)
at Socket.emit (node:events:518:28)
at addChunk (node:internal/streams/readable:561:12)
at readableAddChunkPushByteMode (node:internal/streams/readable:512:3)
at Readable.push (node:internal/streams/readable:392:5)
at TCP.onStreamRead (node:internal/stream_base_commons:189:23)
at TCP.callbackTrampoline (node:internal/async_hooks:130:17) {
severity_local: 'ERROR',
severity: 'ERROR',
code: '23505',
detail: 'Key ("ownerId", checksum)=(4a396719-b723-46b7-9573-980ab3b3f114, \\x1af556693b372bb5ab4b5a5efb472dde114a152d) already exists.',
schema_name: 'public',
table_name: 'assets',
constraint_name: 'UQ_assets_owner_checksum',
file: 'nbtinsert.c',
line: '663',
routine: '_bt_check_unique'
},
sql: 'insert into "assets" ("ownerId", "libraryId", "checksum", "originalPath", "deviceAssetId", "deviceId", "fileCreatedAt", "fileModifiedAt", "localDateTime", "type", "isFavorite", "isArchived", "duration", "isVisible", "originalFileName") values ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15) returning *',
params: [
'4a396719-b723-46b7-9573-980ab3b3f114',
null,
<Buffer 1a f5 56 69 3b 37 2b b5 ab 4b 5a 5e fb 47 2d de 11 4a 15 2d>,
'upload/upload/4a396719-b723-46b7-9573-980ab3b3f114/48/08/48085278-59c9-4543-8def-ad3f003654fe.jpg',
'2009-01-25_-13-39_iPhone-001.jpg-304523',
'CLI',
2013-02-09T16:33:48.713Z,
2013-02-09T16:33:48.713Z,
2013-02-09T16:33:48.713Z,
'IMAGE',
false,
false,
null,
true,
'2009-01-25_-13-39_iPhone-001.jpg'
]
}
I cannot backup my Samsung S24 at all because of this. Is there any known workaround? Completely breaks immich for me sadly...
Query failed : { durationMs: 13.541229002177715, error: PostgresError: duplicate key value violates unique constraint "UQ_assets_owner_checksum" at ErrorResponse (/usr/src/app/node_modules/postgres/cjs/src/connection.js:790:26) at handle (/usr/src/app/node_modules/postgres/cjs/src/connection.js:476:6) at Socket.data (/usr/src/app/node_modules/postgres/cjs/src/connection.js:315:9) at Socket.emit (node:events:518:28) at addChunk (node:internal/streams/readable:561:12) at readableAddChunkPushByteMode (node:internal/streams/readable:512:3) at Readable.push (node:internal/streams/readable:392:5) at TCP.onStreamRead (node:internal/stream_base_commons:189:23) at TCP.callbackTrampoline (node:internal/async_hooks:130:17) { severity_local: 'ERROR', severity: 'ERROR', code: '23505', detail: 'Key ("ownerId", checksum)=(87401635-57fc-4edf-9f8a-898f70437125, \\xfece43fb89984da86e2523deff4d612c5e9a8cdb) already exists.', schema_name: 'public', table_name: 'assets', constraint_name: 'UQ_assets_owner_checksum', file: 'nbtinsert.c', line: '663', routine: '_bt_check_unique' }, sql: 'insert into "assets" ("ownerId", "libraryId", "checksum", "originalPath", "deviceAssetId", "deviceId", "fileCreatedAt", "fileModifiedAt", "localDateTime", "type", "isFavorite", "isArchived", "duration", "isVisible", "originalFileName") values ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15) returning *', params: [ '87401635-57fc-4edf-9f8a-898f70437125', null, <Buffer fe ce 43 fb 89 98 4d a8 6e 25 23 de ff 4d 61 2c 5e 9a 8c db>, 'upload/upload/87401635-57fc-4edf-9f8a-898f70437125/ff/ca/ffca90de-1a3c-4398-85f4-b78dc98610f8.jpg', '1000077622', 'b97976c2180d4e5e156bb496e3ce8b7820dcff0ee0fc4d608596bc8af943407c', 2025-05-11T18:08:06.000Z, 2025-05-11T18:08:09.000Z, 2025-05-11T18:08:06.000Z, 'IMAGE', false, false, '0:00:00.000000', true, '20250511_200806.jpg' ] } [Nest] 18 - 05/18/2025, 7:15:41 PM ERROR [Api:ErrorInterceptor~r8m95iqc] Unknown error: Error: ENOENT: no such file or directory, mkdir 'upload/upload/87401635-57fc-4edf-9f8a-898f70437125/cd/b5' Error: ENOENT: no such file or directory, mkdir 'upload/upload/87401635-57fc-4edf-9f8a-898f70437125/cd/b5' at mkdirSync (node:fs:1364:26) at StorageRepository.mkdirSync (/usr/src/app/dist/repositories/storage.repository.js:130:37) at AssetMediaService.getUploadFolder (/usr/src/app/dist/services/asset-media.service.js:80:32) at /usr/src/app/dist/middleware/file-upload.interceptor.js:130:52 at callbackify (/usr/src/app/dist/middleware/file-upload.interceptor.js:72:31) at FileUploadInterceptor.destination (/usr/src/app/dist/middleware/file-upload.interceptor.js:130:16) at DiskStorage._handleFile (/usr/src/app/node_modules/multer/storage/disk.js:31:8) at FileUploadInterceptor.handleFile (/usr/src/app/dist/middleware/file-upload.interceptor.js:144:29) at /usr/src/app/node_modules/multer/lib/make-middleware.js:139:17 at callbackify (/usr/src/app/dist/middleware/file-upload.interceptor.js:72:16) [Nest] 18 - 05/18/2025, 7:15:51 PM LOG [Api:EventRepository] Websocket Connect: CnHCDMiI9O7H9LDdAAAV
I ran into the same problem while manually uploading images to immich using the api from a python script.
Has anyone here found a workaround yet?
Since there does not seem to be any workaround, did anyone set up 3rd party software to backup the images and ingest them into immich? I heard some might use photosync? Any recommendations? I would prefer to backup with immich but I need a solution.
Same issue happens to me, as well. I have a fresh install of immich and started to backup my android pictures. After around 1k pictures it stopped working with the mentioned duplicate error message.
Have a same issue on the Immich v1.134.0 and iPhone. Failed uploading new photos with duplicate key value violates unique constraint "UQ_assets_owner_checksum":
immich_server | Query failed : {
immich_server | durationMs: 10.70600700378418,
immich_server | error: PostgresError: duplicate key value violates unique constraint "UQ_assets_owner_checksum"
immich_server | at ErrorResponse (/usr/src/app/node_modules/postgres/cjs/src/connection.js:790:26)
immich_server | at handle (/usr/src/app/node_modules/postgres/cjs/src/connection.js:476:6)
immich_server | at Socket.data (/usr/src/app/node_modules/postgres/cjs/src/connection.js:315:9)
immich_server | at Socket.emit (node:events:518:28)
immich_server | at addChunk (node:internal/streams/readable:561:12)
immich_server | at readableAddChunkPushByteMode (node:internal/streams/readable:512:3)
immich_server | at Readable.push (node:internal/streams/readable:392:5)
immich_server | at TCP.onStreamRead (node:internal/stream_base_commons:189:23)
immich_server | at TCP.callbackTrampoline (node:internal/async_hooks:130:17) {
immich_server | severity_local: 'ERROR',
immich_server | severity: 'ERROR',
immich_server | code: '23505',
immich_server | detail: 'Key ("ownerId", checksum)=(050e73ac-7cdb-415e-9694-e4feff3851e1, \\xcef99096c40c24ebee73c14e9bf486e0a3b8bfdf) already exists.',
immich_server | schema_name: 'public',
immich_server | table_name: 'assets',
immich_server | constraint_name: 'UQ_assets_owner_checksum',
immich_server | file: 'nbtinsert.c',
immich_server | line: '663',
immich_server | routine: '_bt_check_unique'
immich_server | },
immich_server | sql: 'insert into "assets" ("ownerId", "libraryId", "checksum", "originalPath", "deviceAssetId", "deviceId", "fileCreatedAt", "fileModifiedAt", "localDateTime", "type", "isFavorite", "duration", "visibility", "originalFileName") values ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14) returning *',
immich_server | params: [
immich_server | '050e73ac-7cdb-415e-9694-e4feff3851e1',
immich_server | null,
immich_server | <Buffer ce f9 90 96 c4 0c 24 eb ee 73 c1 4e 9b f4 86 e0 a3 b8 bf df>,
immich_server | 'upload/upload/050e73ac-7cdb-415e-9694-e4feff3851e1/ea/07/ea077eca-e486-479f-956b-618a1db74230.HEIC',
immich_server | 'A13159D0-AEB0-4E06-9985-94505FDD9A95/L0/001',
immich_server | 'ee27f8f2144e4a029378c722f9df1a2e203e04f501659fdde40eadf9e7c710d6',
immich_server | 2025-06-15T10:15:09.000Z,
immich_server | 2025-06-16T15:50:24.000Z,
immich_server | 2025-06-15T10:15:09.000Z,
immich_server | 'IMAGE',
immich_server | false,
immich_server | '0:00:00.000000',
immich_server | 'timeline',
immich_server | 'IMG_0218.HEIC'
immich_server | ]
immich_server | }
I was also able to reproduce the problem here. If I delete a photo from Immich, the next uploaded photo generates the error message.
immich_server | Query failed : {
immich_server | durationMs: 10.762985000386834,
immich_server | error: PostgresError: duplicate key value violates unique constraint "UQ_assets_owner_checksum"
immich_server | at ErrorResponse (/usr/src/app/node_modules/postgres/cjs/src/connection.js:790:26)
immich_server | at handle (/usr/src/app/node_modules/postgres/cjs/src/connection.js:476:6)
immich_server | at Socket.data (/usr/src/app/node_modules/postgres/cjs/src/connection.js:315:9)
immich_server | at Socket.emit (node:events:518:28)
immich_server | at addChunk (node:internal/streams/readable:561:12)
immich_server | at readableAddChunkPushByteMode (node:internal/streams/readable:512:3)
immich_server | at Readable.push (node:internal/streams/readable:392:5)
immich_server | at TCP.onStreamRead (node:internal/stream_base_commons:189:23)
immich_server | at TCP.callbackTrampoline (node:internal/async_hooks:130:17) {
immich_server | severity_local: 'ERROR',
immich_server | severity: 'ERROR',
immich_server | code: '23505',
immich_server | detail: 'Key ("ownerId", checksum)=(<redacted>, \\x106c33bd54bc43ad96d3de3e319add61fa250071) already exists.',
immich_server | schema_name: 'public',
immich_server | table_name: 'assets',
immich_server | constraint_name: 'UQ_assets_owner_checksum',
immich_server | file: 'nbtinsert.c',
immich_server | line: '663',
immich_server | routine: '_bt_check_unique'
immich_server | },
immich_server | sql: 'insert into "assets" ("ownerId", "libraryId", "checksum", "originalPath", "deviceAssetId", "deviceId", "fileCreatedAt", "fileModifiedAt", "localDateTime", "type", "isFavorite", "duration", "visibility", "livePhotoVideoId", "originalFileName") values ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15) returning *',
immich_server | params: [
immich_server | '<redacted>',
immich_server | null,
immich_server | <Buffer 10 6c 33 bd 54 bc 43 ad 96 d3 de 3e 31 9a dd 61 fa 25 00 71>,
immich_server | 'upload/upload/bfe16f4a-2bdd-466d-9e8c-3dfca2ddafd5/72/69/7269f886-07ab-4d3c-b95e-88223d0cdc5a.HEIC',
immich_server | '0DF89935-39B0-4E2A-8BF6-C0BD203122AB/L0/001',
immich_server | '<redacted>',
immich_server | 2025-06-19T08:22:40.000Z,
immich_server | 2025-06-19T08:22:47.000Z,
immich_server | 2025-06-19T08:22:40.000Z,
immich_server | 'IMAGE',
immich_server | false,
immich_server | '0:00:00.000000',
immich_server | 'timeline',
immich_server | '5b262991-67a2-4fdc-bd6f-e8a0412cc52b',
immich_server | 'IMG_1146.HEIC'
immich_server | ]
immich_server | }
I was even able to reproduce it on an instance of a friend. So this seems to be a problem on many instances. With a clean trashbin the error does not happen. Could you please have a look @alextran1502 ?
I have completely redeployed immich via the officially supported docker compose method, and immediately ran into the problem during the first Android upload. I have tried three different phones (two Pixel devices and one Samsung) and all three ran into this issue. I would really love to use immich, but sadly without a way to sync phones the main use case is kindof broken for me. Again, if anyone has any ideas for a sync app (ideally FOSS) or a workaround please share.
@beposec This indicates that the assets are already on your server. How did you delete the photo? Can you be specific?
Hey @alextran1502, thanks for the fast response! I deleted the image in the Web interface. Then took a new photo on my mobile and opened the Immich App. This produces the error in the log.
Hey @alextran1502 thank you for chiming in! 🙏 Just to let you know, I get this even without deleting a photo first.
@OrbitingOcelot do you already use other means to upload assets to your instance already vie immich-cli or immich-go before uploading from the phone?
@OrbitingOcelot do you already use other means to upload assets to your instance already vie immich-cli or immich-go before uploading from the phone?
@alextran1502 Yes I have used immich-go to upload some older photos, honestly I am not 100 % sure, but I believe there should not have been any overlap between those and the ones to be uploaded from phone directly via immich Android app. Is there any way to clear the upload cache etc. to reset the whole thing?
@OrbitingOcelot Then the app is doing what it is built to do, preventing duplication to get to the server.
I think you are fine. However, if you want to start fresh, that is up to you. You can set the UPLOAD_LOCATION and DB_DATA_LOCATION to a new one