Unicode Decode Error

[hadi-f90]

Problem

In Python 3.12 running pipreqs returns this error:

<unknown>:165: SyntaxWarning: invalid escape sequence '\S'
<unknown>:166: SyntaxWarning: invalid escape sequence '\['
<unknown>:207: SyntaxWarning: invalid escape sequence '\['
<unknown>:456: SyntaxWarning: invalid escape sequence '\S'
<unknown>:200: SyntaxWarning: invalid escape sequence '\:'
Traceback (most recent call last):
  File "/*******/.local/bin/pipreqs", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/*******/.local/lib/python3.12/site-packages/pipreqs/pipreqs.py", line 609, in main
    init(args)
  File "/*******/.local/lib/python3.12/site-packages/pipreqs/pipreqs.py", line 533, in init
    candidates = get_all_imports(
                 ^^^^^^^^^^^^^^^^
  File "/*******/.local/lib/python3.12/site-packages/pipreqs/pipreqs.py", line 136, in get_all_imports
    contents = read_file_content(file_name, encoding)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/*******/.local/lib/python3.12/site-packages/pipreqs/pipreqs.py", line 181, in read_file_content
    contents = f.read()
               ^^^^^^^^
  File "<frozen codecs>", line 322, in decode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb1 in position 81: invalid start byte

Therefore, it doesn't save any requirements.txt

Possible Solution

It seems that changing the afformentioned piece of code into this one can manage to solve it temporarily:

        with open(file_name, "r", encoding=encoding) as f:
            try:
                contents = f.read()
            except UnicodeDecodeError:
                contents = f.read().enode("utf-8")

Although changing this solves the main issue about saving requirements.txt file, it adds raising of a number of warnings about the name and need to check module/lib names from PyPi which slows down pipreqs.

PS: Other changes returned similar warnings

[py563]

+1, same issue on Windows with python 3.13.1 on 0.4 versions of pipreqs too

pipreqs --force --encoding utf-8

(or)

pipreqs --force --encoding utf-8-sig

[janekolszak]

Nope, this doesn't work

[JvdsReform]

I have the same problem

[mzglinski]

Try ignoring the venv folder if you have one, for example

pipreqs --ignore .venv,venv

[Jxlle]

@mzglinski That worked, thanks!!

[HerrFrodo]

I also get this error, setting the encoding and ignoring the venv Folder doesn't work.

[lonswaymar]

Adding that ignoring my virtual environment solved the exact same error.

[sbigstar0310]

Try ignoring the venv folder if you have one, for example

pipreqs --ignore .venv,venv

THANKS 👍