pylint
pylint copied to clipboard
pylint-dev
Suppress SyntaxWarning from invalid escape sequence
Bug description
"""Module docstring"""
from fastcore import script
print(script)
Configuration
No response
Command used
python3.12 -m venv .venv --clear
.venv/bin/python -m pip install pylint fastcore
.venv/bin/pylint file.py
Pylint output
<unknown>:32: SyntaxWarning: invalid escape sequence '\.'
<unknown>:33: SyntaxWarning: invalid escape sequence '\ '
--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)
Expected behavior
No following warnings:
<unknown>:32: SyntaxWarning: invalid escape sequence '\.'
<unknown>:33: SyntaxWarning: invalid escape sequence '\ '
If using Python 3.11, those errors are not raised.
If this is expected behavior, output needs some explanation.
Pylint version
pylint 3.0.3
astroid 3.0.2
Python 3.12.0 (main, Oct 21 2023, 17:44:38) [GCC 9.4.0]
OS / Environment
Ubuntu 20.04.6 LTS
Additional dependencies
astroid==3.0.2
dill==0.3.7
fastcore==1.5.29
isort==5.13.2
mccabe==0.7.0
packaging==23.2
platformdirs==4.1.0
pylint==3.0.3
tomlkit==0.12.3
Extra info
Warning seems to be raised because of these two lines: https://github.com/fastai/fastcore/blob/c9b4c088d3706569c076e7c197c724730be190ab/fastcore/script.py#L32-L33
But it's imported module, so pylint shouldn't raise this warning, right?
pylint uses Python to inspect the code.
As seen in the release notes, this is invalid code and can no longer be executed. Thus, pylint crashes on it as well.
pylintuses Python to inspect the code.As seen in the release notes, this is invalid code and can no longer be executed. Thus,
pylintcrashes on it as well.
Okay, thanks for explanation!
I don't know if this is wanted behavior, but luckily pylint exits with code 0 and still rates my code with grade 10/10.
But for now it's only a SyntaxWarning. It might be 5+ years before it ever becomes a SyntaxError. I'd be in favor of just catching the warning for now in pylint.
Hello you can see what i have when i test there is no Error or Warning
the python_excel.py :
from fastcore import script
print(script)
Do you have same version of fastcore?
What packages do you have? conda list or pip freeze, depending on which one you use. It looks like you are using conda.
yes i have the same version of fastcore i think you can use pyupgrade
You shouldn't need to install fastcore to reproduce the issue. Using a sample file with just the lines linked in the report should be sufficient.
I ran pylint 3.1.0dev0 over the following code and got W1041 (anomalous-backslash-in-string) so it seems this has been fixed in pylint 3? I get no syntax warning on the output. Am I missing something?
import re
def clean_type_str(x:str):
x = str(x)
x = re.sub("(enum |class|function|__main__\.|\ at.*)", '', x)
x = re.sub("(<|>|'|\ )", '', x) # spl characters
return x
Pylint may not be inferring any values in your example. You'll need to test inference.
Just merged https://github.com/pylint-dev/astroid/pull/2380 which should at least improve the situation a bit. Instead of <unknown>, the module name will be included in the warning message.
Tbh I'm not sure if it really makes sense to just ignore all SyntaxWarnings. As an example, most of them I see for Home Assistant, I wouldn't have noticed if not for pylint. If dependencies aren't loaded during the pytest run the warning would never surface.
Users can still choose to ignore them if they want to
python -Wignore:invalid\ escape\ sequence:SyntaxWarning -m pylint ...
# or simply
python -Wi -m pylint ...
As an example, most of them I see for Home Assistant, I wouldn't have noticed if not for pylint.
@cdce8p Do you have a example you can pass along? That seems like a whole batch of false negatives for anomalous-backslash-in-string.
As an example, most of them I see for Home Assistant, I wouldn't have noticed if not for pylint.
@cdce8p Do you have a example you can pass along? That seems like a whole batch of false negatives for
anomalous-backslash-in-string.
Try pip install python-vlc and then run pylint with Python 3.12 over a file with just import vlc. That should do it.
That's the output I get now for python-vlc==3.0.18122 which HA uses currently. AFAIK this hasn't been fixed yet so a newer version should work as well.
# test.py
import vlc
vlc:1180: SyntaxWarning: invalid escape sequence '\w'
vlc:1948: SyntaxWarning: invalid escape sequence '\d'
vlc:1956: SyntaxWarning: invalid escape sequence '\d'
vlc:1964: SyntaxWarning: invalid escape sequence '\d'
vlc:1972: SyntaxWarning: invalid escape sequence '\d'
vlc:2561: SyntaxWarning: invalid escape sequence '\d'
vlc:2574: SyntaxWarning: invalid escape sequence '\d'
vlc:2592: SyntaxWarning: invalid escape sequence '\d'
vlc:2602: SyntaxWarning: invalid escape sequence '\d'
vlc:2716: SyntaxWarning: invalid escape sequence '\l'
vlc:2868: SyntaxWarning: invalid escape sequence '\d'
vlc:2877: SyntaxWarning: invalid escape sequence '\d'
vlc:2919: SyntaxWarning: invalid escape sequence '\l'
vlc:3086: SyntaxWarning: invalid escape sequence '\l'
vlc:3206: SyntaxWarning: invalid escape sequence '\l'
vlc:3425: SyntaxWarning: invalid escape sequence '\d'
vlc:3436: SyntaxWarning: invalid escape sequence '\d'
vlc:3442: SyntaxWarning: invalid escape sequence '\d'
vlc:3463: SyntaxWarning: invalid escape sequence '\d'
vlc:3472: SyntaxWarning: invalid escape sequence '\d'
vlc:3518: SyntaxWarning: invalid escape sequence '\l'
vlc:3622: SyntaxWarning: invalid escape sequence '\@'
vlc:3825: SyntaxWarning: invalid escape sequence '\l'
vlc:3904: SyntaxWarning: invalid escape sequence '\l'
vlc:3911: SyntaxWarning: invalid escape sequence '\l'
vlc:3918: SyntaxWarning: invalid escape sequence '\l'
vlc:3983: SyntaxWarning: invalid escape sequence '\l'
vlc:4583: SyntaxWarning: invalid escape sequence '\d'
vlc:4597: SyntaxWarning: invalid escape sequence '\d'
vlc:4605: SyntaxWarning: invalid escape sequence '\d'
vlc:4613: SyntaxWarning: invalid escape sequence '\d'
vlc:4621: SyntaxWarning: invalid escape sequence '\d'
vlc:4632: SyntaxWarning: invalid escape sequence '\d'
vlc:4664: SyntaxWarning: invalid escape sequence '\d'
vlc:4676: SyntaxWarning: invalid escape sequence '\d'
vlc:4686: SyntaxWarning: invalid escape sequence '\d'
vlc:4696: SyntaxWarning: invalid escape sequence '\d'
vlc:4706: SyntaxWarning: invalid escape sequence '\d'
vlc:4716: SyntaxWarning: invalid escape sequence '\d'
vlc:4732: SyntaxWarning: invalid escape sequence '\d'
vlc:4753: SyntaxWarning: invalid escape sequence '\d'
vlc:4766: SyntaxWarning: invalid escape sequence '\d'
vlc:4781: SyntaxWarning: invalid escape sequence '\d'
vlc:4789: SyntaxWarning: invalid escape sequence '\d'
vlc:4801: SyntaxWarning: invalid escape sequence '\d'
vlc:5546: SyntaxWarning: invalid escape sequence '\l'
vlc:5835: SyntaxWarning: invalid escape sequence '\l'
vlc:6046: SyntaxWarning: invalid escape sequence '\l'
vlc:6188: SyntaxWarning: invalid escape sequence '\l'
vlc:6343: SyntaxWarning: invalid escape sequence '\l'
vlc:6474: SyntaxWarning: invalid escape sequence '\@'
vlc:6746: SyntaxWarning: invalid escape sequence '\l'
vlc:6858: SyntaxWarning: invalid escape sequence '\l'
vlc:6868: SyntaxWarning: invalid escape sequence '\l'
vlc:6878: SyntaxWarning: invalid escape sequence '\l'
vlc:6973: SyntaxWarning: invalid escape sequence '\l'
Ah, that's a great example, since the vlc is not being linted, it's just having its AST built. When I download the vlc.py file and lint it, I get anomalous-backslash-in-string as expected.
Although I don't think we should be swallowing ALL warnings when building the AST or inferring values from third-party code, I do still think in cases where we have a specific known warning that is already covered by a dedicated pylint message, that we shouldn't be polluting pylint's output with it. We don't take responsibility for linting third-party code.
The fact that pylint needs to build the AST or infer a value from third-party code is an implementation detail. When we start preferring .pyi stubs, some of these warnings might go away anyhow.
@crazybolillo, I do still get the warnings in your example, are you testing with Python 3.12?
I was torn myself as it would be trivial to just ignore all SyntaxWarnings when building the AST. However, in case of HA the errors are mainly emitted for parts not covered by pytest. So we wouldn't have known about them otherwise.
Ideally it would be possible to convert them to regular pylint messages (which could be disabled on the import statement). That would require some more effort though as to catch these we would first need to make them SyntaxErrors which could be propagated. Unfortunately that would mean that the parsing will fail (even if it doesn't have to) so it might be necessary to do a second pass.
I think I'm still missing some context -- are you saying there are parts of HA that not only aren't covered by pytest, but aren't covered by pylint? Why not?
I think I'm still missing some context -- are you saying there are parts of HA that not only aren't covered by pytest, but aren't covered by pylint? Why not?
Not HA itself but third party libraries. We only lint our code and pylint is designed to not emit errors for 3rd parties, so it's sometimes difficult to know if something will break if it isn't tested. As example, we recently migrated from 3.11 to 3.12 and our only "standards" were if the dependency would install (some did have build failures) and if the existing tests pass. Everything else was left for users to discover and report via issues.
Maybe that's an area pylint (and ruff) can help. It would require some kind of minimal config but then it might be helpful to run those over all dependencies as well.
Writing this: That would be an argument to ignore the SyntaxWarnings entirely tbh.
Thanks, that's helpful.
I'm thinking we should ignore all the SyntaxWarnings in third-party code, with a caveat that we could check the current state of the warnings filters, and whatever the user has asked to do with SyntaxWarnings, we do that, either ignore, raise, or error.
This is possible since SyntaxWarning is not in the default set of warning filters. This is starting to sound like #9440. Then this lets the linter of HA keep the "accidental feature" of getting some limited linting of 3rd party code. Every pylint user will eventually get "linting of third party code" when the SyntaxWarning eventually becomes a SyntaxError.