Symlink is failing in obscure circumstances

[ildyria]

Discussed in https://github.com/LycheeOrg/Lychee/discussions/3024

https://github.com/LycheeOrg/Lychee/discussions/3024#discussioncomment-12255178 shows PoC that this is working as intended.

However, https://github.com/LycheeOrg/Lychee/discussions/3024#discussioncomment-12256663 shows that it is breaking in their case.

https://github.com/LycheeOrg/Lychee/discussions/3024#discussioncomment-12259587 Traces the failure.

Preliminary Analysis

If you open one of those image via Lychee (go to your-lychee-url/uploads/01/35/filename.jpg, you will know if they are from previous import and need to be "cleaned". There is the php artisan lychee:ghostbuster command for that (it removes any images from the DB without a file, and similarly any file without an image attached, hence why there is a dry run by default.)

Your error is in this function: https://github.com/LycheeOrg/Lychee/blob/master/app/Actions/Photo/Pipes/Standalone/PlacePhoto.php#L65 I suspect around here: https://github.com/LycheeOrg/Lychee/blob/master/app/Actions/Photo/Pipes/Standalone/PlacePhoto.php#L83

if we look at: https://github.com/LycheeOrg/Lychee/blob/master/app/Actions/Photo/Create.php#L170

$pipes = [
			Standalone\FixTimeStamps::class,
			Standalone\InitNamingStrategy::class,
			Shared\HydrateMetadata::class,
			Shared\SetStarred::class,
			Shared\SetParentAndOwnership::class,
			Standalone\SetOriginalChecksum::class,
			Standalone\FetchSourceImage::class,
			Standalone\ExtractGoogleMotionPictures::class,
			Standalone\PlacePhoto::class,
			Standalone\PlaceGoogleMotionVideo::class,
			Standalone\SetChecksum::class,
			Shared\Save::class,
			Standalone\CreateOriginalSizeVariant::class,
			Standalone\CreateSizeVariants::class,
			Standalone\EncodePlaceholder::class,
			Standalone\ReplaceOriginalWithBackup::class,
			Shared\UploadSizeVariantsToS3::class,
		];

		return $this->executePipeOnDTO($pipes, $dto)->getPhoto();

All the operations after PlacePhoto::class as skipped. which means that Shared\Save::class which is in charge of saving the image in your database is not executed, hence the empty albums.

So the question becomes why does the symlink worked in my case and not yours? :thinking:

Question.

Why is this symlink failing?

[d7415]

My best guess (skimming) is that NAS is doing something...

Could try shelling into the container and see if a symlink from the command line has the same behaviour?

[srahimeen]

The NAS is not doing any processing to the images, it just has a shared folder where I store my image files. My server (the machine were Lychee runs) mounts the network drive and can access the images there normally.

I use another image viewing application called Immich where I similarly mounted the shared folder on my NAS as an external volume, and it displays the images properly.

Is there a way to enable some extra logging to get more info on why symlink fails in this case?

[ildyria]

@srahimeen as you can execute docker commands, can you try this: docker exec -it lychee bash to get a terminal inside the container.

And then do ln -s /photography/Images/image1.jpg /var/www/html/Lychee/public/uploads/original/image-link.jpg And then check if you do have the symbolic link ?

[srahimeen]

Gives me an error:

root@71e958b24e95:/# ln -s /photography/Instagram/Tokyo-P400-RICOHTF200-Japan1-2023-02-23-0022-positive-Instagram.jpg /var/www/html/Lychee/public/uploads/original/image-link.jpg
ln: failed to create symbolic link '/var/www/html/Lychee/public/uploads/original/image-link.jpg': Input/output error

However, it does make the file in the uploads/original folder:

[ildyria]

That is an upsetting one. Well at least we know why the step crashes in Lychee. Now to fix it is another story. 🤔

[d7415]

Either don't use symlinks in this case and duplicate the files or we need to keep our own store of external paths. Or I suppose we could see whether hardlinks would work, but that seems like a stretch here. Or people need better NASs 🤷‍♂

[srahimeen]

I am guessing Lychee might have this issue with any network mounted drive? Not sure if it needs the files to exist physically on the same machine Lychee is running on.

[ildyria]

I am guessing Lychee might have this issue with any network mounted drive?

I think at this point this is not a Lychee problem, but more of a Unbuntu/Os problem. We use symlinks to track the original file (and make it accessible to the user on client side), however if that is not supported by the Os that is quite a pickle to solve.

As @d7415 we could try hardlinks, but that will require a bit more engineering. One thing to note is that Lychee does not support directory manipulation on the mounted side. If you add images, that is fine, but if you move image A from folder A to folder B on the mounted/sync folder we cannot track that (Just clarifying this if that is a hard restriction for you).

[srahimeen]

That makes sense.

My server (that is running Lychee) is on Ubuntu 24.04.1 LTS

My NAS is running Synology DSM 7.2.2-72806 Update 3

If you add images, that is fine, but if you move image A from folder A to folder B on the mounted/sync folder we cannot track that

Yes that is fine. The mounted folder just gets images added to it, but images don't really move around once added.