prospector icon indicating copy to clipboard operation
prospector copied to clipboard

[BUG] prospector fails with UnicodeDecodeError on requirements.txt having DOS (CRLF) line endings

Open amai2012 opened this issue 10 months ago • 2 comments

Describe the bug prospector fails with UnicodeDecodeError (see below) if the requirements.txt has DOS line endings.

dos2unix requirements.txt

cures the problem.

To Reproduce

  1. prospector .

Expected behavior Either accept the file - since pip can handle it - or show a helpful error message

Screenshots

Traceback (most recent call last):
  File "/home/me/project/mysite/venv/bin/prospector", line 8, in <module>
    sys.exit(main())
  File "/home/me/project/mysite/venv/lib/python3.10/site-packages/prospector/run.py", line 188, in main
    config = ProspectorConfig()
  File "/home/me/project/mysite/venv/lib/python3.10/site-packages/prospector/config/__init__.py", line 36, in __init__
    self.libraries = self._find_used_libraries(self.config, self.profile)
  File "/home/me/project/mysite/venv/lib/python3.10/site-packages/prospector/config/__init__.py", line 215, in _find_used_libraries
    for found_dep in autodetect_libraries(self.workdir):
  File "/home/me/project/mysite/venv/lib/python3.10/site-packages/prospector/autodetect.py", line 89, in autodetect_libraries
    libraries = find_from_requirements(path)
  File "/home/me/project/mysite/venv/lib/python3.10/site-packages/prospector/autodetect.py", line 72, in find_from_requirements
    reqs = find_requirements(path)
  File "/home/me/project/mysite/venv/lib/python3.10/site-packages/requirements_detector/detect.py", line 83, in find_requirements
    requirements += from_requirements_txt(reqfile)
  File "/home/me/project/mysite/venv/lib/python3.10/site-packages/requirements_detector/detect.py", line 146, in from_requirements_txt
    for req in f.readlines():
  File "/usr/lib/python3.10/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte
> file requirements.txt
requirements.txt: Unicode text, UTF-16, little-endian text, with CRLF line terminators

Environment (please complete the following information):

  • OS: Linux (Ubuntu 22.04)
  • Tool: any?
  • Prospector version: 1.10.3
  • Python version: 3.10

Additional context dependencies installed, e.g. the output of pip freeze also helps.

doesn't matter imho

amai2012 avatar Apr 25 '24 08:04 amai2012