astroid
astroid copied to clipboard
pylint-dev
infer results of open() and file() calls
Needed for PyCQA/pylint#1245.
It’s a very rough draft, input will be appreciated.
Coverage decreased (-0.1%) to 89.251% when pulling 1436e39fd048699b8f84d172d1dfefde72ca94a5 on rogalski:infer-open-calls into 9f8e6f074ba7d9d060355a67a22c30a0c2cb44fd on PyCQA:master.
Coverage decreased (-0.1%) to 89.251% when pulling 67f992c74e95d5c1e0930497ce519f577374c5f3 on rogalski:infer-open-calls into 9f8e6f074ba7d9d060355a67a22c30a0c2cb44fd on PyCQA:master.
Coverage decreased (-0.1%) to 89.251% when pulling ba9dff896aea967125da66d9b52e51fa978a9bd0 on rogalski:infer-open-calls into 9f8e6f074ba7d9d060355a67a22c30a0c2cb44fd on PyCQA:master.
Coverage decreased (-0.3%) to 89.068% when pulling ba9dff896aea967125da66d9b52e51fa978a9bd0 on rogalski:infer-open-calls into 9f8e6f074ba7d9d060355a67a22c30a0c2cb44fd on PyCQA:master.
Looks good, but check my suggestion to see if it can improve the situation for Python 3.
@PCManticore most definitely, Python3 will be improved, thanks for your comments!
@PCManticore
How would I reasonably infer arguments of function call? Covering only kwargs/only args/unpacking/defaults etc. will be tricky. Is CallSite something which may be used here?
Coverage decreased (-0.05%) to 89.318% when pulling 0432a6315e647e38ade645ccbdf68ac2c4871ffc on rogalski:infer-open-calls into 9f8e6f074ba7d9d060355a67a22c30a0c2cb44fd on PyCQA:master.
It needs more work, but conceptually it's much closer to final patch comparing to previous one.
Coverage drop is due to guard clauses not being hit.
Coverage decreased (-0.04%) to 89.326% when pulling 98fe5c8dd2ef305bb728cb4ce91bfc8fd21ff088 on rogalski:infer-open-calls into 9f8e6f074ba7d9d060355a67a22c30a0c2cb44fd on PyCQA:master.
Coverage increased (+0.07%) to 89.326% when pulling 24180efab9bcb3f593bc5f9888f153b4d9dcc8df on rogalski:infer-open-calls into 7b5a3bfb99d7a4554f64e3529daaf3275052cf74 on PyCQA:master.
Quick question - why does astroid seems to be unable to fill arguments in inferred value for open() function? Inferred function seems to be correct - at least doc parameter is copied from proper built-in function.
inspect module seems to retrieve this signature easily, it looks like there's no reason for AST to not contain these information.
>>> print(next(astroid.extract_node('open').infer()).args)
Arguments(vararg=None,
kwarg=None,
args=None,
defaults=[],
kwonlyargs=[],
kw_defaults=[],
annotations=[],
varargannotation=None,
kwargannotation=None)
>>> import inspect
>>> inspect.signature(open)
<Signature (file, mode='r', buffering=-1, encoding=None, errors=None, newline=None, closefd=True, opener=None)>
>>>
Coverage increased (+0.04%) to 89.297% when pulling 97d8fde555baf76105460155fdfeb0913a7c303d on rogalski:infer-open-calls into 7b5a3bfb99d7a4554f64e3529daaf3275052cf74 on PyCQA:master.
Because it is a builtin and we have limited support for inferring builtins. Check raw_building, where everything happens (and also check the rewrite from 2.0; unfortunately, in 2.0, we cannot infer open() at all, but that is due to a bug). inspect.signature is not a good comparison, since it is inspecing __text_signature__ for builtins. astroid might access this in the future, since, for builtins, we are operating with the actual objects, so we can take this hints into account.
@PCManticore I can either hardcode information on open() signature in astroid code, or use inspect module to try to fill all objects built in raw_builder.py. For me, second option is much better than the first, but I guess it should target 2.0 branch, given huge rewrite was merged in that branch. On the same time, it's such a "big" change that you may be reluctant to do right now.
I see three options:
- hard-code information on open signature in astroid code, merge changes in pylint and astroid, postpone implementing generic feature of inspecting live objects to build more rich AST
- start implementing using instect to build more rich AST (which should target 2.0 branch)
- mark PRs as blocked and start working on them again after 2.0 is released
What you think about it?
I think we can wait for 2.0, if you don't mind. We can just do the work once, in 2.0's raw_building.py, rather than duplicating it for 1.5 and , then, for 2.0.
While the code is there for quite some time, after pylint 2.0, I want to have more often releases, so this should be out pretty fast.
Note that I rewrote raw_building.py in 2.0 to use inspect, including for building the builtin modules. You should look at the result of that code and start from there, since this is target 2.0 anyways.
@rogalski, It's a complete rewrite, so many different commits. As @PCManticore said, the 2.0 branch holds the current version of raw_building.py.
I'm going to close this as there are now many merge conflicts and the reviewers are no longer active within the project. It would probably be better to start a new PR if anybody wants to tackle this.
