upscayl [BUG][PATH PARSING] Cannot upscayl Non-ASCII path

Checklist

[X] I have tried ALL of the steps mentioned in the troubleshooting guide.
[X] I have searched for this in the issues tab.
[ ] My error involves vkEnumeratePhysicalDevices or vkCreateInstance, which means I should ignore the other issues and file this one despite the last checkbox.

Describe the Bug

If the path that contains the destination folder, or the folder itself, has non-ascii characters in the name, the program fails to save and show the enlarged image. The logs say the image has been processed correctly and report it as saved in the selected path, but the image is not saved (or shown in the UI) unless an output folder without non-ascii characters in the name or path is selected. The log does not display any errors. By all accounts, the image should be saved but it's not, and the UI does not display it.

To Reproduce

Set the output to a folder called lámpara or cañón.
Try to upscale any image.

Upscayl Version (or commit hash)

2.10.9 and 2.11

Platform

Linux

OS Version

Arch Linux (using both AUR and AppImage)

GPU Name

AMD Lexa PRO (Radeon RX 550X rev. c7)

Expected Behavior

The image should be saved correctly in any folder with write permissions.

Screenshots

imagen

Logs

⚙️ Setting saveImageAs to png
🔀 Setting model to realesrgan-x4plus
⚙️ Setting gpuId to empty string
🔀 Setting model to 
🔀 Setting model to realesrgan-x4plus
🔄 Resetting image paths
⤵️ Dropped file:  {"type":"image/png","filePath":"/home/andi/ñ/Captura de pantalla de 2024-04-29 12-50-27.png","extension":"png"}
🖼 Setting image path:  /home/andi/ñ/Captura de pantalla de 2024-04-29 12-50-27.png
🗂 Setting output path:  /home/andi/ñ
🖼 imagePath:  /home/andi/ñ/Captura de pantalla de 2024-04-29 12-50-27.png
🔤 Extension:  png
🔄 Resetting Upscaled Image Path
🏁 UPSCAYL
𝌖 BACKEND REPORTED:  🖼️ Updating No Image Processing:  false
𝌖 BACKEND REPORTED:  📐 Updating Compression:  0
𝌖 BACKEND REPORTED:  Is Default Model? :  true
𝌖 BACKEND REPORTED:  ✅ Upscayl Variables:  {"model":"realesrgan-x4plus","gpuId":null,"saveImageAs":"png","inputDir":"/home/andi/ñ","outputDir":"/home/andi/ñ","fullfileName":"Captura de pantalla de 2024-04-29 12-50-27.png","fileName":"Captura de pantalla de 2024-04-29 12-50-27","scale":"4","outFile":"/home/andi/ñ/Captura de pantalla de 2024-04-29 12-50-27_upscayl_4x_realesrgan-x4plus.png","compression":0}
𝌖 BACKEND REPORTED:  📢 Upscayl Command:  -i,/home/andi/ñ/Captura de pantalla de 2024-04-29 12-50-27.png,-o,/home/andi/ñ/Captura de pantalla de 2024-04-29 12-50-27_upscayl_4x_realesrgan-x4plus.png,-s 4,-m,/opt/upscayl/models,-n,realesrgan-x4plus,-f,png
𝌖 BACKEND REPORTED:  👶 Updating Child Processes:  {"binary":"/opt/upscayl/bin/upscayl-bin","args":["/opt/upscayl/bin/upscayl-bin","-i","/home/andi/ñ/Captura de pantalla de 2024-04-29 12-50-27.png","-o","/home/andi/ñ/Captura de pantalla de 2024-04-29 12-50-27_upscayl_4x_realesrgan-x4plus.png","-s 4","-m","/opt/upscayl/models","-n","realesrgan-x4plus","-f","png"]}
𝌖 BACKEND REPORTED:  🛑 Updating Stopped:  false
𝌖 BACKEND REPORTED:  image upscayl:  🚀 Starting Upscayl - Copyright © 2024
✨ Detected scale x4

🚧 UPSCAYL_PROGRESS:  🚀 Starting Upscayl - Copyright © 2024
✨ Detected scale x4

𝌖 BACKEND REPORTED:  image upscayl:  [0 AMD Radeon RX 550 / 550 Series (RADV POLARIS12)]  queueC=1[4]  queueG=0[1]  queueT=0[1]
[0 AMD Radeon RX 550 / 550 Series (RADV POLARIS12)]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[0 AMD Radeon RX 550 / 550 Series (RADV POLARIS12)]  fp16-p/s/a=1/1/0  int8-p/s/a=1/1/1
[0 AMD Radeon RX 550 / 550 Series (RADV POLARIS12)]  subgroup=64  basic=1  vote=1  ballot=1  shuffle=1
[1 AMD Radeon Graphics (RADV RENOIR)]  queueC=1[4]  queueG=0[1]  queueT=0[1]
[1 AMD Radeon Graphics (RADV RENOIR)]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[1 AMD Radeon Graphics (RADV RENOIR)]  fp16-p/s/a=1/1/1  int8-p/s/a=1/1/1
[1 AMD Radeon Graphics (RADV RENOIR)]  subgroup=64  basic=1  vote=1  ballot=1  shuffle=1

🚧 UPSCAYL_PROGRESS:  [0 AMD Radeon RX 550 / 550 Series (RADV POLARIS12)]  queueC=1[4]  queueG=0[1]  queueT=0[1]
[0 AMD Radeon RX 550 / 550 Series (RADV POLARIS12)]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[0 AMD Radeon RX 550 / 550 Series (RADV POLARIS12)]  fp16-p/s/a=1/1/0  int8-p/s/a=1/1/1
[0 AMD Radeon RX 550 / 550 Series (RADV POLARIS12)]  subgroup=64  basic=1  vote=1  ballot=1  shuffle=1
[1 AMD Radeon Graphics (RADV RENOIR)]  queueC=1[4]  queueG=0[1]  queueT=0[1]
[1 AMD Radeon Graphics (RADV RENOIR)]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[1 AMD Radeon Graphics (RADV RENOIR)]  fp16-p/s/a=1/1/1  int8-p/s/a=1/1/1
[1 AMD Radeon Graphics (RADV RENOIR)]  subgroup=64  basic=1  vote=1  ballot=1  shuffle=1

𝌖 BACKEND REPORTED:  image upscayl:  0.00%

🚧 UPSCAYL_PROGRESS:  0.00%

𝌖 BACKEND REPORTED:  image upscayl:  2.50%

🚧 UPSCAYL_PROGRESS:  2.50%

𝌖 BACKEND REPORTED:  image upscayl:  5.00%

🚧 UPSCAYL_PROGRESS:  5.00%

𝌖 BACKEND REPORTED:  image upscayl:  7.50%

🚧 UPSCAYL_PROGRESS:  7.50%

𝌖 BACKEND REPORTED:  image upscayl:  10.00%

🚧 UPSCAYL_PROGRESS:  10.00%

𝌖 BACKEND REPORTED:  image upscayl:  12.50%

🚧 UPSCAYL_PROGRESS:  12.50%

𝌖 BACKEND REPORTED:  image upscayl:  15.00%

🚧 UPSCAYL_PROGRESS:  15.00%

𝌖 BACKEND REPORTED:  image upscayl:  17.50%

🚧 UPSCAYL_PROGRESS:  17.50%

𝌖 BACKEND REPORTED:  image upscayl:  20.00%

🚧 UPSCAYL_PROGRESS:  20.00%

𝌖 BACKEND REPORTED:  image upscayl:  22.50%

🚧 UPSCAYL_PROGRESS:  22.50%

𝌖 BACKEND REPORTED:  image upscayl:  25.00%

🚧 UPSCAYL_PROGRESS:  25.00%

𝌖 BACKEND REPORTED:  image upscayl:  27.50%

🚧 UPSCAYL_PROGRESS:  27.50%

𝌖 BACKEND REPORTED:  image upscayl:  30.00%

🚧 UPSCAYL_PROGRESS:  30.00%

𝌖 BACKEND REPORTED:  image upscayl:  32.50%

🚧 UPSCAYL_PROGRESS:  32.50%

𝌖 BACKEND REPORTED:  image upscayl:  35.00%

🚧 UPSCAYL_PROGRESS:  35.00%

𝌖 BACKEND REPORTED:  image upscayl:  37.50%

🚧 UPSCAYL_PROGRESS:  37.50%

𝌖 BACKEND REPORTED:  image upscayl:  40.00%

🚧 UPSCAYL_PROGRESS:  40.00%

𝌖 BACKEND REPORTED:  image upscayl:  42.50%

🚧 UPSCAYL_PROGRESS:  42.50%

𝌖 BACKEND REPORTED:  image upscayl:  45.00%

🚧 UPSCAYL_PROGRESS:  45.00%

𝌖 BACKEND REPORTED:  image upscayl:  47.50%

🚧 UPSCAYL_PROGRESS:  47.50%

𝌖 BACKEND REPORTED:  image upscayl:  50.00%

🚧 UPSCAYL_PROGRESS:  50.00%

𝌖 BACKEND REPORTED:  image upscayl:  52.50%

🚧 UPSCAYL_PROGRESS:  52.50%

𝌖 BACKEND REPORTED:  image upscayl:  55.00%

🚧 UPSCAYL_PROGRESS:  55.00%

𝌖 BACKEND REPORTED:  image upscayl:  57.50%

🚧 UPSCAYL_PROGRESS:  57.50%

𝌖 BACKEND REPORTED:  image upscayl:  60.00%

🚧 UPSCAYL_PROGRESS:  60.00%

𝌖 BACKEND REPORTED:  image upscayl:  62.50%

🚧 UPSCAYL_PROGRESS:  62.50%

𝌖 BACKEND REPORTED:  image upscayl:  65.00%

🚧 UPSCAYL_PROGRESS:  65.00%

𝌖 BACKEND REPORTED:  image upscayl:  67.50%

🚧 UPSCAYL_PROGRESS:  67.50%

𝌖 BACKEND REPORTED:  image upscayl:  70.00%

🚧 UPSCAYL_PROGRESS:  70.00%

𝌖 BACKEND REPORTED:  image upscayl:  72.50%

🚧 UPSCAYL_PROGRESS:  72.50%

𝌖 BACKEND REPORTED:  image upscayl:  75.00%

🚧 UPSCAYL_PROGRESS:  75.00%

𝌖 BACKEND REPORTED:  image upscayl:  77.50%

🚧 UPSCAYL_PROGRESS:  77.50%

𝌖 BACKEND REPORTED:  image upscayl:  80.00%

🚧 UPSCAYL_PROGRESS:  80.00%

𝌖 BACKEND REPORTED:  image upscayl:  82.50%

🚧 UPSCAYL_PROGRESS:  82.50%

𝌖 BACKEND REPORTED:  image upscayl:  85.00%

🚧 UPSCAYL_PROGRESS:  85.00%

𝌖 BACKEND REPORTED:  image upscayl:  87.50%

🚧 UPSCAYL_PROGRESS:  87.50%

𝌖 BACKEND REPORTED:  image upscayl:  90.00%

🚧 UPSCAYL_PROGRESS:  90.00%

𝌖 BACKEND REPORTED:  image upscayl:  92.50%

🚧 UPSCAYL_PROGRESS:  92.50%

𝌖 BACKEND REPORTED:  image upscayl:  95.00%

🚧 UPSCAYL_PROGRESS:  95.00%

𝌖 BACKEND REPORTED:  image upscayl:  97.50%

🚧 UPSCAYL_PROGRESS:  97.50%

𝌖 BACKEND REPORTED:  image upscayl:  🏞️ Resizing image according to output scale

🚧 UPSCAYL_PROGRESS:  🏞️ Resizing image according to output scale

𝌖 BACKEND REPORTED:  image upscayl:  🏞️ Resized image from 784x413 to 3136x1652

🚧 UPSCAYL_PROGRESS:  🏞️ Resized image from 784x413 to 3136x1652

𝌖 BACKEND REPORTED:  💯 Done upscaling
𝌖 BACKEND REPORTED:  ♻ Scaling and converting now...
upscaledImagePath:  /home/andi/ñ/Captura%20de%20pantalla%20de%202024-04-29%2012-50-27_upscayl_4x_realesrgan-x4plus.png
💯 UPSCAYL_DONE:  /home/andi/ñ/Captura%20de%20pantalla%20de%202024-04-29%2012-50-27_upscayl_4x_realesrgan-x4plus.png
⚙️ Getting saveImageAs from localStorage:  png
⚙️ Getting model from localStorage:  {"label":"General Photo (Real-ESRGAN)","value":"realesrgan-x4plus"}
⚙️ Setting gpuId to empty string

Apr 29 '24 18:04 conradolandia

Huh, interesting. #770 mentions the same bug but I was unable to reproduce it.

Using lámpara as the directory name caused the error on Linux.

On MacOS, It's not the case, the image upscayls just fine.

Apr 30 '24 07:04 NayamAmarshe

Huh, interesting. #770 mentions the same bug but I was unable to reproduce it.

Using lámpara as the directory name caused the error on Linux.

On MacOS, It's not the case, the image upscayls just fine.

I think it was very hasty to close my thread, and I don't understand why. I have tested on two different computers, with different versions of Tumbleweed (6 months without updating one of them) and the problem occurred on both of them. Also, I was previously using Upscayl without problems in folders with non-ASCII characters, so if suddenly after an update of Upscayl, it stops working properly, I think it is clear that some change in Upscayl causes the problem, and even more when all the other applications work in those folders with the same files without problems.

For my part, I have again taken the trouble to check from which version some change in the Upscayl code caused this problem to occur and I have found out that it is the versions after 2.9.8 that fail. Interestingly, version 2.9.9 at least gave an error (not like the latest version, which tells the user that ‘everything is fine’). In 2.9.9 it shows this error:

🚧 UPSCAYL_PROGRESS:  93.33%

🐞 BACKEND REPORTED:  💯 Done upscaling
🐞 BACKEND REPORTED:  ♻ Scaling and converting now...
🐞 BACKEND REPORTED:  📐 Processing Image:  {"originalWidth":860,"originalHeight":500,"customWidth":null,"scale":"3","saveImageAs":"png","compressionPercentage":0,"compressionLevel":0}
🐞 BACKEND REPORTED:  🖼️ Checking if original image exists:  /#áçñ#/cartel_pequeña.webp
🐞 BACKEND REPORTED:  ❌ Error processing (scaling and converting) the image. Please report this error on GitHub. Error: Input file is missing: /#áçñ#/cartel_pequeña_upscayl_3x_realesrgan-x4plus.png

It shows an error about Upscayl is not able to find same file was processing.

I even tried to test the install version (RPM) but I couldn't, because it doesn't find the libuuid library (in openSUSE, it's libuuid1).

Of course, in any way I can help you find out what's going on by providing more information or testing, please let me know.

Apr 30 '24 08:04 RafaelLinux

@RafaelLinux I only closed the other thread because I was unable to reproduce the issue with your provided paths on Linux. This issue however, mentioned another name which I was able to reproduce the bug with.

Apr 30 '24 09:04 NayamAmarshe

I'll investigate more

Apr 30 '24 09:04 NayamAmarshe

Interestingly, version 2.9.9 at least gave an error (not like the latest version, which tells the user that ‘everything is fine’).

That's because in 2.9.9 we pointed a separate tool to the file path to post-process it. We later infused processing into our own CLI which tries to output it all in one go.

Apr 30 '24 13:04 aaronliu0130

I expect it helps in some way.

Apr 30 '24 17:04 RafaelLinux

@dosu-bot

May 29 '24 17:05 NayamAmarshe

@NayamAmarshe I'm here to help you with any bugs, questions, or contributions while you wait for a human maintainer.

Upscayl fails to save and display upscaled images when the output folder contains non-ASCII characters on Linux due to issues with path handling. The sanitizePath function is used to handle paths, ensuring they are properly sanitized to avoid errors from special characters. Here’s how it works:

Normalize the file path: Replaces backslashes (\) with forward slashes (/).
Split the file path: Splits the normalized file path into segments.
Encode each segment: Encodes each segment using encodeURIComponent.
Join the encoded segments: Joins the encoded segments back together with forward slashes.
Return the final file URL: Returns the encoded file path.

This function ensures that non-ASCII characters are percent-encoded, which should handle most cases correctly ^[1].

Additionally, logging and debugging information in versions 2.10.9 and 2.11 can provide more insight. The logit function logs various stages of the upscaling process, including errors and progress updates. This can help diagnose issues related to non-ASCII characters in the output folder path ^[2]^[3].

For further investigation, ensure that all paths, including the output folder path, are sanitized using the sanitizePath function before being used in the application. This will help avoid issues with non-ASCII characters in paths ^[3].

Details

About Dosu This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

May 29 '24 17:05 dosubot[bot]

This bot has completely ignored the fact that this problem has arisen from an update of Upscayl and did not happen before under the same conditions. Therefore, mentioning to the end user that a possible solution is to sanitise the route using certain functions is totally absurd. Only the developer really knows what changes there have been between versions to cause something to fail now that worked before.

May 29 '24 19:05 RafaelLinux

Hey @RafaelLinux valid feedback! We want to support and are actively working on improving Dosu's triage bugs via recent changes, so stay tuned :)

May 30 '24 02:05 devstein

Therefore, mentioning to the end user that a possible solution is to sanitise the route using certain functions is totally absurd.

I don't think it did that. It only mentioned me and I called the bot to investigate the issue for me.

May 30 '24 13:05 NayamAmarshe

It seems that setting the output folder to a non-ASCII path works. The problem is with upscayl-ncnn somewhere and it' strange.

It's able to read the input directory but not able to write to the same directory with non-ASCII directory. Also, the file being non-ASCII does not seem to matter, it's only the directory here that's causing the issue.

(gdb) run -i ./lámpara/#áçèïñ.jpg -o ./lámpara/hrllo.png -m ./resources/models/ -n 'realesrgan-x4fast' -f png
Starting program: /home/user/Public/upscayl/resources/linux/bin/upscayl-bin -i ./lámpara/#áçèïñ.jpg -o ./lámpara/hrllo.png -m ./resources/models/ -n 'realesrgan-x4fast' -f png
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
🚀 Starting Upscayl - Copyright © 2024
✨ Detected scale x4
✨ Using the default scale x4
[0 NVIDIA GeForce GTX 1660 SUPER]  queueC=2[8]  queueG=0[16]  queueT=1[2]
[0 NVIDIA GeForce GTX 1660 SUPER]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[0 NVIDIA GeForce GTX 1660 SUPER]  fp16-p/s/a=1/1/1  int8-p/s/a=1/1/1
[0 NVIDIA GeForce GTX 1660 SUPER]  subgroup=32  basic=1  vote=1  ballot=1  shuffle=1
[1 llvmpipe (LLVM 15.0.7, 256 bits)]  queueC=0[1]  queueG=0[1]  queueT=0[1]
[1 llvmpipe (LLVM 15.0.7, 256 bits)]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[1 llvmpipe (LLVM 15.0.7, 256 bits)]  fp16-p/s/a=1/1/1  int8-p/s/a=1/1/1
[1 llvmpipe (LLVM 15.0.7, 256 bits)]  subgroup=8  basic=1  vote=1  ballot=1  shuffle=1
[New Thread 0x7fffe5bff640 (LWP 22640)]
[New Thread 0x7fffe53fe640 (LWP 22641)]
[New Thread 0x7fffe4bfd640 (LWP 22642)]
[New Thread 0x7fffd7fff640 (LWP 22644)]
[New Thread 0x7fffcffff640 (LWP 22671)]
[New Thread 0x7fffcf7fe640 (LWP 22672)]
[New Thread 0x7fffceffd640 (LWP 22673)]
[New Thread 0x7fffce7fc640 (LWP 22674)]
[New Thread 0x7fffcdffb640 (LWP 22675)]
[New Thread 0x7fffcd7fa640 (LWP 22676)]
[New Thread 0x7fffccff9640 (LWP 22677)]
[New Thread 0x7fffc7fff640 (LWP 22678)]
[New Thread 0x7fffc77fe640 (LWP 22679)]
[New Thread 0x7fffc6ffd640 (LWP 22680)]
[New Thread 0x7fffc67fc640 (LWP 22681)]
[New Thread 0x7fffc5ffb640 (LWP 22682)]
[New Thread 0x7fffc57fa640 (LWP 22709)]
[New Thread 0x7fffc4ff9640 (LWP 22710)]
[New Thread 0x7fffbffff640 (LWP 22711)]
[New Thread 0x7fffbf7fe640 (LWP 22712)]
[New Thread 0x7fffbeffd640 (LWP 22713)]
[Thread 0x7fffbffff640 (LWP 22711) exited]
[Thread 0x7fffc57fa640 (LWP 22709) exited]
0.00%
25.00%
100.00%
50.00%
75.00%
100.00%
[Thread 0x7fffbeffd640 (LWP 22713) exited]
[Thread 0x7fffc4ff9640 (LWP 22710) exited]

Thread 21 "upscayl-bin" received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffbf7fe640 (LWP 22712)]
__pthread_kill_implementation (no_tid=0, signo=6, threadid=140736406218304) at ./nptl/pthread_kill.c:44
44      ./nptl/pthread_kill.c: No such file or directory.

May 30 '24 13:05 NayamAmarshe

backtrace:

(gdb) backtrace
#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=140736406218304) at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=140736406218304) at ./nptl/pthread_kill.c:78
#2  __GI___pthread_kill (threadid=140736406218304, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
#3  0x00007ffff7c42476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4  0x00007ffff7c287f3 in __GI_abort () at ./stdlib/abort.c:79
#5  0x0000555555599247 in save(void*) [clone .cold] ()
#6  0x00007ffff7c94ac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#7  0x00007ffff7d26850 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

The error is happening in the save(void*) function.

May 30 '24 13:05 NayamAmarshe

@aaronliu0130 He's busy, I'll be fixing the code :)

May 31 '24 20:05 NayamAmarshe

The issue was with Upscayl NCNN. To fix the issue, I did these things:

Set the locale as empty, which means default locale settings for paths.
Replace wstring on *nix to just string.

Jun 01 '24 10:06 NayamAmarshe

Also discovered a bug in Nautilus where dragging a file with # in the name into the terminal causes the # to disappear.

Jun 01 '24 10:06 NayamAmarshe