Decode error during scan some files.

[changlexc]

I think there should be an exception handler but not failed all the process. /home/CHANGLE/cve-bin-tool-main/cve_bin_tool/version_scanner.py:116 in is_executable 116 output = subprocess.check_output(["file", filename]).decode(

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe6 in position 72: invalid continuation byte

[M-Faheem-Khan]

Hey @changlexc, it seem that the file is encoded using UTF-8 which is returning the issue. @terriko do you think allowing the user choose encoding as a flag is a better option than adding a try/except here? IMHO adding both is a good idea. If you believe so as well can I take this as a task?

Scope:

• Adding a try/except around the output(where file is being read)
• Adding new flag: users can set their own encoding type

[changlexc]

For you information, I just add a try except on my local file to workaround this issue. try: output = subprocess.check_output(["file", filename]).decode( sys.stdout.encoding ) except: self.logger.warning(f"Failure decoding {filename}") output = f"{filename}: cannot open"

[terriko]

Putting in a try-except sounds like a good first solution. That's what we do with other files we can't open for whatever reason.

I agree that allowing user to provide encoding might be a nice option to have, but python defaults to utf-8 in a lot of places so it's going to be a non-trivial thing to provide and test correctly so that no error messages have weird behaviour and whatnot.

@M-Faheem-Khan if you want to give it a shot, feel free, but be warned that it's likely to be a pretty finicky and time-consuming feature to add! I don't even want to think about what an interface for handling multiple encodings for files in a whole directory, so you'd probably want to focus on the single file or single encoding for all files in directory/archive options first.