Filename/path contains unicode characters were skipped in "Recover Files"

[tatsaa]

Hi, When I trying to recover a batch of missing files ,some successfully recovered,while some not. I realized that if filename/path contains unicode characters(e.g. Japanese),it was skipped. Think add unicode characters filename/path support would be quite useful. Using Windows 10 Pro 21H1 - OS Build 19043.1237 Elucidate 2020.9.1022.1 Snapraid 11.5 Here's the log:

09:52:33.2032 [29] INFO: ######################################### 09:52:33.2032 [29] INFO: Using: c:\SnapRAID\SnapRAID.exe 09:52:33.2032 [29] INFO: with: -c "c:\SnapRAID\SnapRAID.conf" fix -f "/test 시험 测试 テスト/5GHz_Wireless_Channels.png" 09:52:33.2592 [34] INFO: StdOut[Self test...] 09:52:33.3652 [34] INFO: StdOut[Loading state from D:/snapraid.content...] 09:52:38.0702 [34] INFO: StdOut[Searching disk d1...] 09:52:38.1672 [34] INFO: StdOut[Searching disk d2...] 09:52:38.3202 [34] INFO: StdOut[Searching disk d3...] 09:52:38.3822 [34] INFO: StdOut[Searching disk d4...] 09:52:38.5302 [34] INFO: StdOut[Selecting...] 09:52:38.5342 [34] INFO: StdOut[Using 1817 MiB of memory for the file-system.] 09:52:38.5342 [34] INFO: StdOut[Initializing...] 09:52:38.5342 [34] INFO: StdOut[Selecting...] 09:52:41.8422 [34] INFO: StdOut[Fixing...] 09:52:41.9242 [34] INFO: StdOut[Nothing to do] 09:52:41.9242 [34] INFO: StdOut[Everything OK] 09:52:42.0562 [34] INFO: StdOut[] 09:52:42.0562 [33] DEBUG: Process has exited 09:52:42.0562 [29] INFO: ExitCode=[0]

[Smurf-IV]

Hello, Thanks for the report..

09:52:33.2032 [29] INFO: Using: c:\SnapRAID\SnapRAID.exe 09:52:33.2032 [29] INFO: with: -c "c:\SnapRAID\SnapRAID.conf" fix -f "/test 시험 测试 テスト/5GHz_Wireless_Channels.png"

1. Are the charcaters correct in the command line that has been logged out ?
2. What happens if you run the above from a command line ?

[tatsaa]

sorry for the delay. I tested it in Windows CMD console, unfortunately Unicode characters does not show properly ,and keeps failing while trying to recover the specific file. the logs is here:

c:\snapraid>snapraid check -m Self test... Loading state from D:/snapraid.content... Searching disk d1... Searching disk d2... Searching disk d3... Searching disk d4... Selecting... Scanning disk d1... Scanning disk d2... Scanning disk d3... Scanning disk d4... Using 1793 MiB of memory for the file-system. Initializing... Selecting... Checking... recoverable "test 鞁滍棙 娴嬭瘯 銉嗐偣銉?5GHz_Wireless_Channels.png" 100% completed, 1 MB accessed in 0:00

   1 errors
   0 unrecoverable errors

WARNING! There are errors!

c:\snapraid>snapraid -c "c:\SnapRAID\SnapRAID.conf" fix -f "/test 시험 测试 テスト/5GHz_Wireless_Channels.png" Self test... Loading state from D:/snapraid.content... Searching disk d1... Searching disk d2... Searching disk d3... Searching disk d4... Selecting... Using 1793 MiB of memory for the file-system. Initializing... Selecting... Fixing... Nothing to do Everything OK

c:\snapraid>snapraid -c "c:\SnapRAID\SnapRAID.conf" fix -f "test 鞁滍棙 娴嬭瘯 銉嗐偣銉?5GHz_Wireless_Channels.png" Self test... Loading state from D:/snapraid.content... Searching disk d1... Searching disk d2... Searching disk d3... Searching disk d4... Selecting... Using 1793 MiB of memory for the file-system. Initializing... Selecting... Fixing... Nothing to do Everything OK

c:\snapraid>snapraid -c "c:\SnapRAID\SnapRAID.conf" fix -m Self test... Loading state from D:/snapraid.content... Searching disk d1... Searching disk d2... Searching disk d3... Searching disk d4... Selecting... Scanning disk d1... Scanning disk d2... Scanning disk d3... Scanning disk d4... Using 1793 MiB of memory for the file-system. Initializing... Selecting... Fixing... recovered "test 鞁滍棙 娴嬭瘯 銉嗐偣銉?5GHz_Wireless_Channels.png" 100% completed, 1 MB accessed in 0:00

   1 errors
   1 recovered errors
   0 unrecoverable errors

Everything OK

[Smurf-IV]

I think the problem is in the snapRaid then.. Can you ask on their system please ?

[tatsaa]

thank you. After searching in snapraid forum, I found the problem was posted since 2013 !

[gristooki]

Has there been any development regarding this issue? I have a large number of files with unicode names, and I'd really like to use SnapRaid.

[Smurf-IV]

Hello @gristooki

1. Are you saying this problem is in Elucidate, or in the Command line when sent direct to SnapRaid ?
2. Can you list the references to the problem (2013 onwards) here please ?

[gristooki]

I haven't tried it with command line, but when I tested recovering some deleted files using Elucidate, I was unable to recover any files with Japanese characters. Files without Japanese characters recovered perfectly. As for the references to the problem referred to above, I'm unsure of specifics since it was someone else who wrote that, but they may have been talking about this thread I found with a cursory Google search: https://sourceforge.net/p/snapraid/discussion/1677233/thread/93ca231a/

[Smurf-IV]

@gristooki If you look back through this thread (to the comment on on Oct 7, 2021) you will see that the problem is on the command line to SnapRaid. Can you attempt the same steps with the version of SnapRaid that you currently have installed. Also state what versions of SnapRiad and Elucidate you are using please ?

[gristooki]

It is 12.2 (latest) and I just tried it on command line. Same result as above. It seems like it is a SnapRaid issue.

[Smurf-IV]

OK, thanks for confirming.

[ztc8069]

If filename/path contains other languages(like chinese Japanese)，it made Garbled Code in program running，causing invalid parity data. fortunately when I run snapraid via conlose（cmd/powershell）, I can use "chcp 65001" command line to set conlose run in "UTF-8 mode",it can slove that's problem. but I can't run Elucidate in "UTF-8 mode",can you slove it?

[Smurf-IV]

use chcp 65001 command line to set console run in UTF-8 mode,it can solve that's problem. but I can't run Elucidate in "UTF-8 mode",can you solve it?

Great find, I'll look into this the next time I update Elucidate