csv-diff-report icon indicating copy to clipboard operation
csv-diff-report copied to clipboard

Encoding::CompatibilityError when file path has non-ASCII character under some environment

Open scivola opened this issue 5 years ago • 0 comments

This exception may occur under, for example, Japanese Windows environment.

In the case, the encoding of the string got via ARGV may be not UTF-8 but Windows 31J (a.k.a. Codepage 932).

# demo1.rb

p ARGV.first.encoding
> ruby demo1.rb foo
#<Encoding:Windows-31J>

Generally, two strings which do not contain any non-ASCII character can be joined, even if one is in UTF-8 and the other is in Windows 31J.

p ["foo".encode("UTF-8"), "bar".encode("CP932")].join
# Not raised.

However, if these strings contain a non-ASCII character, Encoding::CompatibilityError occurs.

p ["あ".encode("UTF-8"), "う".encode("CP932")].join
# => `join': incompatible character encodings: UTF-8 and Windows-31J (En
coding::CompatibilityError)

So, if one of the file paths and a cell value of the CSV have any non-ASCII character, the same exception may occur here.

I think that converting the file paths into UTF-8 may solve the problem.

scivola avatar Nov 23 '18 02:11 scivola