pylint
pylint copied to clipboard
pylint-dev
Fatal error astroid.exceptions.AstroidError
Bug description
Probably related to https://github.com/PyCQA/pylint/issues/6986
When parsing the following file: https://github.com/ddnet/ddnet/blob/f8dccb8a122aff04b7b779157d342c265665b5c7/datasrc/compile.py
The code is from this PR https://github.com/ddnet/ddnet/pull/5210
Command used
The job we use in our CI is (https://github.com/ddnet/ddnet/actions/runs/2548613924/workflow):
- name: Pylint
run: |
pylint --version
find . -type f -name "*.py" -not -path './ddnet-libs/*' -print0 | xargs -0 pylint
with this config: https://github.com/ddnet/ddnet/blob/master/.pylintrc
Pylint output
The trace can also be found in our CI: https://github.com/ddnet/ddnet/runs/7022094695?check_suite_focus=true
pylint crashed with a AstroidError and with the following stacktrace:
Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/pylint/checkers/utils.py", line 1258, in safe_infer
value = next(infer_gen)
File "/usr/lib/python3.10/site-packages/astroid/nodes/node_ng.py", line 168, in infer
yield from self._infer(context=context, **kwargs)
File "/usr/lib/python3.10/site-packages/astroid/decorators.py", line 136, in raise_if_nothing_inferred
yield next(generator)
File "/usr/lib/python3.10/site-packages/astroid/decorators.py", line 105, in wrapped
for res in _func(node, context, **kwargs):
File "/usr/lib/python3.10/site-packages/astroid/bases.py", line 134, in _infer_stmts
for inf in stmt.infer(context=context):
File "/usr/lib/python3.10/site-packages/astroid/nodes/node_ng.py", line 182, in infer
for i, result in enumerate(generator):
File "/usr/lib/python3.10/site-packages/astroid/decorators.py", line 136, in raise_if_nothing_inferred
yield next(generator)
File "/usr/lib/python3.10/site-packages/astroid/decorators.py", line 105, in wrapped
for res in _func(node, context, **kwargs):
File "/usr/lib/python3.10/site-packages/astroid/nodes/node_ng.py", line 182, in infer
for i, result in enumerate(generator):
File "/usr/lib/python3.10/site-packages/astroid/decorators.py", line 136, in raise_if_nothing_inferred
yield next(generator)
File "/usr/lib/python3.10/site-packages/astroid/decorators.py", line 105, in wrapped
for res in _func(node, context, **kwargs):
File "/usr/lib/python3.10/site-packages/astroid/inference.py", line 481, in _filter_operation_errors
for result in infer_callable(self, context):
File "/usr/lib/python3.10/site-packages/astroid/inference.py", line 902, in _infer_augassign
for lhs, rhs in itertools.product(lhs_iter, rhs_iter):
File "/usr/lib/python3.10/site-packages/astroid/bases.py", line 134, in _infer_stmts
for inf in stmt.infer(context=context):
File "/usr/lib/python3.10/site-packages/astroid/nodes/node_ng.py", line 182, in infer
for i, result in enumerate(generator):
File "/usr/lib/python3.10/site-packages/astroid/decorators.py", line 136, in raise_if_nothing_inferred
yield next(generator)
File "/usr/lib/python3.10/site-packages/astroid/decorators.py", line 105, in wrapped
for res in _func(node, context, **kwargs):
File "/usr/lib/python3.10/site-packages/astroid/nodes/node_ng.py", line 182, in infer
for i, result in enumerate(generator):
File "/usr/lib/python3.10/site-packages/astroid/decorators.py", line 136, in raise_if_nothing_inferred
yield next(generator)
File "/usr/lib/python3.10/site-packages/astroid/decorators.py", line 105, in wrapped
for res in _func(node, context, **kwargs):
File "/usr/lib/python3.10/site-packages/astroid/inference.py", line 481, in _filter_operation_errors
for result in infer_callable(self, context):
File "/usr/lib/python3.10/site-packages/astroid/inference.py", line 902, in _infer_augassign
for lhs, rhs in itertools.product(lhs_iter, rhs_iter):
File "/usr/lib/python3.10/site-packages/astroid/bases.py", line 134, in _infer_stmts
for inf in stmt.infer(context=context):
File "/usr/lib/python3.10/site-packages/astroid/nodes/node_ng.py", line 182, in infer
for i, result in enumerate(generator):
File "/usr/lib/python3.10/site-packages/astroid/decorators.py", line 136, in raise_if_nothing_inferred
yield next(generator)
File "/usr/lib/python3.10/site-packages/astroid/decorators.py", line 105, in wrapped
for res in _func(node, context, **kwargs):
File "/usr/lib/python3.10/site-packages/astroid/nodes/node_ng.py", line 182, in infer
for i, result in enumerate(generator):
File "/usr/lib/python3.10/site-packages/astroid/decorators.py", line 136, in raise_if_nothing_inferred
yield next(generator)
File "/usr/lib/python3.10/site-packages/astroid/decorators.py", line 105, in wrapped
for res in _func(node, context, **kwargs):
File "/usr/lib/python3.10/site-packages/astroid/inference.py", line 481, in _filter_operation_errors
for result in infer_callable(self, context):
File "/usr/lib/python3.10/site-packages/astroid/inference.py", line 902, in _infer_augassign
for lhs, rhs in itertools.product(lhs_iter, rhs_iter):
File "/usr/lib/python3.10/site-packages/astroid/bases.py", line 134, in _infer_stmts
for inf in stmt.infer(context=context):
File "/usr/lib/python3.10/site-packages/astroid/nodes/node_ng.py", line 182, in infer
for i, result in enumerate(generator):
File "/usr/lib/python3.10/site-packages/astroid/decorators.py", line 136, in raise_if_nothing_inferred
yield next(generator)
File "/usr/lib/python3.10/site-packages/astroid/decorators.py", line 105, in wrapped
for res in _func(node, context, **kwargs):
File "/usr/lib/python3.10/site-packages/astroid/nodes/node_ng.py", line 182, in infer
for i, result in enumerate(generator):
File "/usr/lib/python3.10/site-packages/astroid/decorators.py", line 136, in raise_if_nothing_inferred
yield next(generator)
File "/usr/lib/python3.10/site-packages/astroid/decorators.py", line 105, in wrapped
for res in _func(node, context, **kwargs):
File "/usr/lib/python3.10/site-packages/astroid/inference.py", line 481, in _filter_operation_errors
for result in infer_callable(self, context):
This part is repeated a lot... so many times I could not post it because of github limits
File "/usr/lib/python3.10/site-packages/astroid/inference.py", line 902, in _infer_augassign
for lhs, rhs in itertools.product(lhs_iter, rhs_iter):
File "/usr/lib/python3.10/site-packages/astroid/bases.py", line 134, in _infer_stmts
for inf in stmt.infer(context=context):
File "/usr/lib/python3.10/site-packages/astroid/nodes/node_ng.py", line 182, in infer
for i, result in enumerate(generator):
File "/usr/lib/python3.10/site-packages/astroid/decorators.py", line 136, in raise_if_nothing_inferred
yield next(generator)
File "/usr/lib/python3.10/site-packages/astroid/decorators.py", line 105, in wrapped
for res in _func(node, context, **kwargs):
File "/usr/lib/python3.10/site-packages/astroid/nodes/node_ng.py", line 182, in infer
for i, result in enumerate(generator):
File "/usr/lib/python3.10/site-packages/astroid/decorators.py", line 136, in raise_if_nothing_inferred
yield next(generator)
File "/usr/lib/python3.10/site-packages/astroid/decorators.py", line 105, in wrapped
for res in _func(node, context, **kwargs):
File "/usr/lib/python3.10/site-packages/astroid/inference.py", line 481, in _filter_operation_errors
for result in infer_callable(self, context):
File "/usr/lib/python3.10/site-packages/astroid/inference.py", line 902, in _infer_augassign
for lhs, rhs in itertools.product(lhs_iter, rhs_iter):
File "/usr/lib/python3.10/site-packages/astroid/bases.py", line 134, in _infer_stmts
for inf in stmt.infer(context=context):
File "/usr/lib/python3.10/site-packages/astroid/nodes/node_ng.py", line 182, in infer
for i, result in enumerate(generator):
File "/usr/lib/python3.10/site-packages/astroid/decorators.py", line 136, in raise_if_nothing_inferred
yield next(generator)
File "/usr/lib/python3.10/site-packages/astroid/decorators.py", line 105, in wrapped
for res in _func(node, context, **kwargs):
File "/usr/lib/python3.10/site-packages/astroid/nodes/node_ng.py", line 182, in infer
for i, result in enumerate(generator):
File "/usr/lib/python3.10/site-packages/astroid/decorators.py", line 136, in raise_if_nothing_inferred
yield next(generator)
File "/usr/lib/python3.10/site-packages/astroid/decorators.py", line 105, in wrapped
for res in _func(node, context, **kwargs):
File "/usr/lib/python3.10/site-packages/astroid/inference.py", line 481, in _filter_operation_errors
for result in infer_callable(self, context):
File "/usr/lib/python3.10/site-packages/astroid/inference.py", line 902, in _infer_augassign
for lhs, rhs in itertools.product(lhs_iter, rhs_iter):
File "/usr/lib/python3.10/site-packages/astroid/bases.py", line 134, in _infer_stmts
for inf in stmt.infer(context=context):
File "/usr/lib/python3.10/site-packages/astroid/nodes/node_ng.py", line 182, in infer
for i, result in enumerate(generator):
File "/usr/lib/python3.10/site-packages/astroid/decorators.py", line 136, in raise_if_nothing_inferred
yield next(generator)
RecursionError: maximum recursion depth exceeded
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 727, in _check_file
check_astroid_module(ast_node)
File "/usr/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 926, in check_astroid_module
retval = self._check_astroid_module(
File "/usr/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 976, in _check_astroid_module
walker.walk(node)
File "/usr/lib/python3.10/site-packages/pylint/utils/ast_walker.py", line 93, in walk
self.walk(child)
File "/usr/lib/python3.10/site-packages/pylint/utils/ast_walker.py", line 93, in walk
self.walk(child)
File "/usr/lib/python3.10/site-packages/pylint/utils/ast_walker.py", line 93, in walk
self.walk(child)
File "/usr/lib/python3.10/site-packages/pylint/utils/ast_walker.py", line 90, in walk
callback(astroid)
File "/usr/lib/python3.10/site-packages/pylint/checkers/refactoring/recommendation_checker.py", line 172, in visit_for
self._check_consider_using_dict_items(node)
File "/usr/lib/python3.10/site-packages/pylint/checkers/refactoring/recommendation_checker.py", line 254, in _check_consider_using_dict_items
iterating_object_name = utils.get_iterating_dictionary_name(node)
File "/usr/lib/python3.10/site-packages/pylint/checkers/utils.py", line 1601, in get_iterating_dictionary_name
inferred = safe_infer(node.iter)
File "/usr/lib/python3.10/site-packages/pylint/checkers/utils.py", line 1262, in safe_infer
raise AstroidError from e
astroid.exceptions.AstroidError
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 684, in _check_files
self._check_file(get_ast, check_astroid_module, file)
File "/usr/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 729, in _check_file
raise astroid.AstroidError from e
astroid.exceptions.AstroidError
Expected behavior
pylint should not crash at this file At least I think my changes at our python files are correct (they do at least their jobs correct).
Pylint version
local:
pylint 2.14.1
astroid 2.11.5
Python 3.10.5 (main, Jun 6 2022, 18:49:26) [GCC 12.1.0]
CI:
pylint 2.4.4
astroid 2.3.3
Python 3.8.10 (default, Mar 15 2022, 12:22:08)
both report this error
### OS / Environment
Local: Arch Linux 5.18.2-arch1-1
CI: ubuntu-20.04
@C0D3D3V Is this crash also occurring locally? Your CI is using a very outdated version of pylint, so it might be caused by a whole range of issues fixed between pylint 2.4.4 and 2.14.3.
If so, could you provide the full crash report pylint locally produces or update pylint in your CI so we can get a trace for an updated pylint and astroid version.
Is this crash also occurring locally
yes, the trace is from my local pylint
yes, the trace is from my local pylint
Could you then also provide some of the other information that is in the crash log, pylint should tell you where you can find it after the traceback has been printed. Specifically, knowing on which file and on which node pylint crashes would be useful to know.
Mh must somehow got lost. I added it, but had to remove a lot of things because it was to long for github. I probably removed it accidentally. pylint-crash-2022-06-23-15.txt terminal_output.txt
It fails on the file I linked in line 297 https://github.com/ddnet/ddnet/blob/41523fca8e10bac438a74f43466bd3861a748f01/datasrc/compile.py#L297
Oh wow. Well, I know what's the cause of this. The 30+ assignments to lines trip up astroid as it tries to infer what lines can be. It needs to check every assignment to be sure.
Why don't you just use one assignment? That would be better for performance as well I think.
We shouldn't crash on this but you can probably do:
lines = [
'#include "protocol.h"',
'#include <engine/shared/packer.h>',
'#include <engine/shared/protocol.h>',
'#include <engine/shared/uuid_manager.h>',
'#include <game/mapitems_ex.h>',
]
or
lines = '''
#include "protocol.h"
#include <engine/shared/packer.h>
#include <engine/shared/protocol.h>
#include <engine/shared/uuid_manager.h>
#include <game/mapitems_ex.h>
'''
:)
Thank you guys :D I took the opportunity that the crash gave me to refactor our 12-14 year old python file a little bit.
https://github.com/ddnet/ddnet/pull/5210/commits/9d309a50329a382ceaa5766045722f99dcb8f949
now pylint no longer crashes
No problem @C0D3D3V , thank you for opening the issue !
It looks like astroid trigger a RecursionError for genuinely long inference call in this case. I think we could move this issue to astroid, so we catch that and re-raise an InferenceError, what do you think @DanielNoord ?
I'd like opinion of others. I'm no expert on RecursionError but I would imagine that we need something like if try_count == max_inferred_values: catch_error() which would add a lot of unnecessary if statements for very niche cases.
Hello,
I've just encountered an error that could very well be linked to this one. I managed to spot the exact part of the code that is causing the crash, so maybe that could help.
Here is the faulty snippet:
class Color(NamedTuple):
RED,CYAN,BLUE,BLACK,GREEN,WHITE = 31,36,34,30,32,37
pylint crashed with a AstroidError and with the following stacktrace:
Traceback (most recent call last):
File "/home/.../.cache/pypoetry/virtualenvs/create-project-wqycC8Vp-py3.10/lib/python3.10/site-packages/astroid/inference_tip.py", line 37, in _inference_tip_cached
result = _cache[func, node]
KeyError: (<function infer_typing_namedtuple_class at 0x7fbdf7d555a0>, <ClassDef.Color l.16 at 0x7fbdf7851120>)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/.../.cache/pypoetry/virtualenvs/create-project-wqycC8Vp-py3.10/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 727, in _check_file
check_astroid_module(ast_node)
File "/home/.../.cache/pypoetry/virtualenvs/create-project-wqycC8Vp-py3.10/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 926, in check_astroid_module
retval = self._check_astroid_module(
File "/home/.../.cache/pypoetry/virtualenvs/create-project-wqycC8Vp-py3.10/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 976, in _check_astroid_module
walker.walk(node)
File "/home/.../.cache/pypoetry/virtualenvs/create-project-wqycC8Vp-py3.10/lib/python3.10/site-packages/pylint/utils/ast_walker.py", line 93, in walk
self.walk(child)
File "/home/.../.cache/pypoetry/virtualenvs/create-project-wqycC8Vp-py3.10/lib/python3.10/site-packages/pylint/utils/ast_walker.py", line 90, in walk
callback(astroid)
File "/home/.../.cache/pypoetry/virtualenvs/create-project-wqycC8Vp-py3.10/lib/python3.10/site-packages/pylint/checkers/base/basic_checker.py", line 498, in visit_functiondef
self._check_dangerous_default(node)
File "/home/.../.cache/pypoetry/virtualenvs/create-project-wqycC8Vp-py3.10/lib/python3.10/site-packages/pylint/checkers/base/basic_checker.py", line 513, in _check_dangerous_default
value = next(default.infer())
File "/home/.../.cache/pypoetry/virtualenvs/create-project-wqycC8Vp-py3.10/lib/python3.10/site-packages/astroid/nodes/node_ng.py", line 168, in infer
yield from self._infer(context=context, **kwargs)
File "/home/.../.cache/pypoetry/virtualenvs/create-project-wqycC8Vp-py3.10/lib/python3.10/site-packages/astroid/decorators.py", line 136, in raise_if_nothing_inferred
yield next(generator)
File "/home/.../.cache/pypoetry/virtualenvs/create-project-wqycC8Vp-py3.10/lib/python3.10/site-packages/astroid/decorators.py", line 105, in wrapped
for res in _func(node, context, **kwargs):
File "/home/.../.cache/pypoetry/virtualenvs/create-project-wqycC8Vp-py3.10/lib/python3.10/site-packages/astroid/inference.py", line 308, in infer_attribute
for owner in self.expr.infer(context):
File "/home/.../.cache/pypoetry/virtualenvs/create-project-wqycC8Vp-py3.10/lib/python3.10/site-packages/astroid/nodes/node_ng.py", line 182, in infer
for i, result in enumerate(generator):
File "/home/.../.cache/pypoetry/virtualenvs/create-project-wqycC8Vp-py3.10/lib/python3.10/site-packages/astroid/decorators.py", line 136, in raise_if_nothing_inferred
yield next(generator)
File "/home/.../.cache/pypoetry/virtualenvs/create-project-wqycC8Vp-py3.10/lib/python3.10/site-packages/astroid/decorators.py", line 105, in wrapped
for res in _func(node, context, **kwargs):
File "/home/.../.cache/pypoetry/virtualenvs/create-project-wqycC8Vp-py3.10/lib/python3.10/site-packages/astroid/bases.py", line 134, in _infer_stmts
for inf in stmt.infer(context=context):
File "/home/.../.cache/pypoetry/virtualenvs/create-project-wqycC8Vp-py3.10/lib/python3.10/site-packages/astroid/nodes/node_ng.py", line 158, in infer
results = list(self._explicit_inference(self, context, **kwargs))
File "/home/.../.cache/pypoetry/virtualenvs/create-project-wqycC8Vp-py3.10/lib/python3.10/site-packages/astroid/inference_tip.py", line 44, in _inference_tip_cached
result = _cache[func, node] = list(func(*args, **kwargs))
File "/home/.../.cache/pypoetry/virtualenvs/create-project-wqycC8Vp-py3.10/lib/python3.10/site-packages/astroid/brain/brain_namedtuple_enum.py", line 483, in infer_typing_namedtuple_class
generated_class_node.locals[attr] = class_node.locals[attr]
KeyError: 'tuple'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/.../.cache/pypoetry/virtualenvs/create-project-wqycC8Vp-py3.10/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 684, in _check_files
self._check_file(get_ast, check_astroid_module, file)
File "/home/.../.cache/pypoetry/virtualenvs/create-project-wqycC8Vp-py3.10/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 729, in _check_file
raise astroid.AstroidError from e
astroid.exceptions.AstroidError
Replacing this code with a declaration line by line resolved my problem:
class Color(NamedTuple):
RED = 31
CYAN = 36
BLUE = 34
BLACK = 30
GREEN = 32
WHITE = 37
I'm using VSC and pylint is running on each save, if that may matter....
Cheers
Thanks @Blind4Basics, that looks to be a different root cause for the AstroidError (not having to do with recursion). Could you open a new issue? (I couldn't reproduce a problem with your example as written because it doesn't have the definition of NamedTuple.)
~Not to my knowledge, although we did fix https://github.com/PyCQA/astroid/issues/1646 (different cause)~
Ah, looking at the solution in PyCQA/astroid#1705, it was general enough to fix this, yes.