UnicodeDecodeError not caught & filename giving issues not reported.

[GadgetSteve]

Describe the issue Running a scan and found in an otherwise working file that UnicodeDecodeError was raised crashing checkov.

Examples Eventually found which file was causing the issue, as below, and that the yaml file contained:

    # ℹCommand-line programs to run using the OS shell.
    # 📚 https://git.io/JvXDl

    # ✏️ If the Autobuild fails

Luckly all in comments!

Exception Trace Please share the trace for the exception and all relevant output by checkov. To maximize the understanding, please run checkov with LOG_LEVEL set to debug as follows:

LOG_LEVEL=DEBUG checkov ...

Not available!

Desktop (please complete the following information):

• OS: Windows 10-64
• Checkov Version: 3.2.85
• Python Version: 3.12.3

Additional context Was able to identify the problem file by modifying C:\temp\test_install_checkov\Lib\site-packages\checkov\github_actions\runner.py line 67-69 From:

            if not file_content:
                with open(f, 'r') as f_obj:
                    file_content = f_obj.read()

To:

            if not file_content:
                with open(f, 'r') as f_obj:
                    try:
                        file_content = f_obj.read()
                    except UnicodeDecodeError as err:
                        print(err)
                        print("Attempting to read:", f)
                        return None

This allowed me to actually see which file was giving the issues & since the odd characters were all in comments I was able to remove them so as to complete the run.

In general it is a shame that the UnicodeDecodeError does not give the name of the file that was being read when the exception occurred but a review of the code so as to improve reporting can allow the users to address such problems. (Or maybe add errors='xmlcharrefreplace' or errors='backslashreplace' on all open operations?)

[itai1357]

Hi @GadgetSteve

first of all, thanks a lot for reporting the issue. if I understood correctly, this is the content of the problematic file

    # ℹCommand-line programs to run using the OS shell.
    # 📚 https://git.io/JvXDl

    # ✏️ If the Autobuild fails

am i right?

can you tell what the path of the file is can you share the command you use to run?

in addition, if you find your changes as useful for other users, you are welcoming to contribute (-:

[GadgetSteve]

That was a part of the content, the lines that gave the problems. The problem happened in a work environment, so I can not share the actual file. After 2 months, I can't recall the exact command, but I know that I was scanning a large number of files.

Regards Steve

[stale[bot]]

Thanks for contributing to Checkov! We've automatically marked this issue as stale to keep our issues list tidy, because it has not had any activity for 6 months. It will be closed in 14 days if no further activity occurs. Commenting on this issue will remove the stale tag. If you want to talk through the issue or help us understand the priority and context, feel free to add a comment or join us in the Checkov slack channel at codifiedsecurity.slack.com Thanks!

[stale[bot]]

Closing issue due to inactivity. If you feel this is in error, please re-open, or reach out to the community via slack: codifiedsecurity.slack.com Thanks!