pylint
pylint copied to clipboard
pylint-dev
import-error on python core modules compiled files
Current behavior
The software repository (entware-ng), I am using, has recently changed the python packages to only include compiled files. This seems to prevent pylint from properly checking imported core modules causing unnecessary import errors.
pylint errors
E: 2, 0: Unable to import '__future__' (import-error)
E: 3, 0: Unable to import 'os' (import-error)
E: 4, 0: Unable to import 'logging' (import-error)
E: 5, 0: Unable to import 'configparser' (import-error)
snippet example
# -*- coding: utf-8 -*-
from __future__ import absolute_import
import os
import logging
from configparser import ConfigParser
python3.6 default path
Python 3.6.1 (default, Apr 20 2017, 07:46:35)
[GCC 6.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print(sys.path)
['', '/opt/lib/python36.zip', '/opt/lib/python3.6', '/opt/lib/python3.6/lib-dynload', '/opt/lib/python3.6/site-packages']
>>>
python3.6 modules
$ find /opt/lib/python3.6/ -type f | grep -E '/opt/lib/python3.6/(__future__|os|logging|configparser)'
/opt/lib/python3.6/configparser.pyc
/opt/lib/python3.6/os.pyc
/opt/lib/python3.6/logging/__init__.pyc
/opt/lib/python3.6/logging/config.pyc
/opt/lib/python3.6/logging/handlers.pyc
/opt/lib/python3.6/__future__.pyc
Expected behavior
Ability to test import on compiled files or at least to identify this case and generated an informational message instead of current import error.
pylint --version output
pylint 1.7.1, astroid 1.5.2 Python 3.6.1 (default, Apr 20 2017, 07:46:35) [GCC 6.3.0]
Ability to test import on compiled files I'm quite sure you cannot build AST tree from bytecode file which means Pylint won't be able to do that.
an informational message instead of current import error I don't see much value, I'd assume that Pylint won't be able to do much without AST trees of standard library anyway.
@rogalski Thanks for your quick response. I was hoping for a way to ignore these specific errors without ignoring all import errors. I guess if pylint cannot differentiate this case, then there is no way to determine if an import error is real or a false positive.
You mentioned that pylint won't be able to do much without AST trees of standard library. Will this affect other tests besides the one highlighted in this issue?
You may disable those inline, although I'd just ignore all of them in pylintrc file. I'm just not sure what new problems would arise due to lack of those stdlib files.
There was too many to manually disable inline so I had to disable the import-error message in the rc file.
As far as I can see, the only check that is no longer triggering is the unused-import one on standard libraries only which would make sense. Other than that I don't see any other issues.
Maybe adding a configuration setting in the imports section to prevent from checking standard libraries would be the solution?
For me, it would be polluting config section with option that will be almost never used.
ignore-import-errors-on-stdlib? The point is those modules exist everywhere. Why there should be a switch for that?
Maybe maintainer will disagree...
Sorry to comment on a closed issue, but I'm confused by the explanation here. Certainly, you can't create an AST from a .pyc file, but how does that mean that you flag import os as "cannot import os module"? Checking if you can import a module just needs a check that the module exists, not that it's possible to parse it.
I'm very aware that people saying "why is this so hard" on issues are (a) usually wrong, and (b) very annoying, but I'd really like to understand why this can't be done. I'm trying to bundle a standalone copy of pylint using the "embedded distribution" of Python for Windows, and that distribution triggers this issue. So if there's a relatively simple explanation of what the difficulty is here that I'm not understanding, then I'd be very grateful to know.
@pfmoore No worries, thank you for asking. While for the import-error check we could detect if the file exists and move on, pylint checks won't be useful at all if they are not able to build an AST for imported modules, especially for stdlib ones like os. The difficulty does not pertain to the actual code changes but rather to what expectancies pylint has for modules that should be available in source form.
A possible solution would be to use --disable=import-error or an alternative could be detecting if the module is compiled only in which case we should not emit the import-error message.
+1 on not emitting the import-error when the module is compiled - when using an editor like VS Code, the current behaviour means that lines that are actually fine are incorrectly highlighed as errors, which is not ideal.
+1 for a way to disable these warnings separately. I am using 3rd party libraries which are shipped only as .pyc files in almost all of our scripts and it would be great to remove just those warnings.
Since it actually confused me for a while, why I was able to import the packages, but pylint was not, I think it would be valuable to have a descriptive warning for this case.
Is this still being worked on? I also have the issue where I'm using 3rd party libraries that are shipped only as .pyc files.
@joshveea it seems no one had time to work on this. Feel free to send a PR!
There has been some work in astroid to support living objects and compiled files.
That said, excluding .pyc files from import-error seems like it would be a nice fix, that shouldn't necessarily be high effort.
@DanielNoord Maybe you meant something like that, but wouldn't it be relative easy to treat compiled python files like c-extensions and do runtime inspection? I tried it with a simple patch in manager.py in astroid and the results where great. A complete solution should probably distinguish between modules from the standard library and other stuff and consider the --extension-pkg-whitelist option.
@mgidde I won't be very active in the next couple of weeks, but a PR that shows this behaviour in astroid would be much appreciated I think. astroid PRs tend to take a little longer to get merged due to the lower number of active maintainers, but often they are some of the most valuable contributions to pylint.
@DanielNoord Thanks for your answer, I will try to find to find some time to provide a suitable PR. But don't hold your breath, these are busy days ;)
Hello!,
I was wondering if there is a workaround or a solution. Currently, this seems to be a problem for our team, and I haven't been able to go around it.
