pylint
pylint copied to clipboard
pylint-dev
InconsistentMroError: Cannot create a consistent method resolution order for MROs
Steps to reproduce
mkdir test
cd test
virtualenv pyt
pyt/bin/pip install setuptools==26.1.1 zc.buildout==1.7.1
# Add files buildout.cfg and test.py
pyt/bin/buildout
./bin/pylint test.py
Where: buildout.cfg
[buildout]
index = https://pypi.python.org/simple/
extends = https://dist.plone.org/release/4-latest/versions.cfg
parts = test
versions = versions
[test]
recipe = zc.recipe.egg
eggs =
plone.dexterity
Products.CMFPlone
pylint
[versions]
pylint = 1.9.3
test.py
# -*- coding: utf-8 -*-
from plone.dexterity.content import Container
class MyContainer(Container):
var = 1
def set_var(self, value):
self.var = value
Current behavior
InconsistentMroError: Cannot create a consistent method resolution order for MROs:
Traceback (most recent call last):
File "/home/user/cache-python/eggs/pylint-1.9.3-py2.7.egg/pylint/lint.py", line 948, in get_ast
return MANAGER.ast_from_file(filepath, modname, source=True)
File "/home/user/cache-python/eggs/astroid-1.6.5-py2.7.egg/astroid/manager.py", line 80, in ast_from_file
return AstroidBuilder(self).file_build(filepath, modname)
File "/home/user/cache-python/eggs/astroid-1.6.5-py2.7.egg/astroid/builder.py", line 153, in file_build
return self._post_build(module, encoding)
File "/home/user/cache-python/eggs/astroid-1.6.5-py2.7.egg/astroid/builder.py", line 173, in _post_build
self.delayed_assattr(delayed)
File "/home/user/cache-python/eggs/astroid-1.6.5-py2.7.egg/astroid/builder.py", line 239, in delayed_assattr
if not _can_assign_attr(inferred, node.attrname):
File "/home/user/cache-python/eggs/astroid-1.6.5-py2.7.egg/astroid/builder.py", line 82, in _can_assign_attr
slots = node.slots()
File "/home/user/cache-python/eggs/astroid-1.6.5-py2.7.egg/astroid/decorators.py", line 28, in cached
cache[func] = result = func(*args, **kwargs)
File "/home/user/cache-python/eggs/astroid-1.6.5-py2.7.egg/astroid/scoped_nodes.py", line 2591, in slots
slots = list(grouped_slots())
File "/home/user/cache-python/eggs/astroid-1.6.5-py2.7.egg/astroid/scoped_nodes.py", line 2576, in grouped_slots
for cls in self.mro()[:-1]:
File "/home/user/cache-python/eggs/astroid-1.6.5-py2.7.egg/astroid/scoped_nodes.py", line 2671, in mro
return self._compute_mro(context=context)
File "/home/user/cache-python/eggs/astroid-1.6.5-py2.7.egg/astroid/scoped_nodes.py", line 2642, in _compute_mro
mro = base._compute_mro(context=context)
File "/home/user/cache-python/eggs/astroid-1.6.5-py2.7.egg/astroid/scoped_nodes.py", line 2642, in _compute_mro
mro = base._compute_mro(context=context)
File "/home/user/cache-python/eggs/astroid-1.6.5-py2.7.egg/astroid/scoped_nodes.py", line 2642, in _compute_mro
mro = base._compute_mro(context=context)
File "/home/user/cache-python/eggs/astroid-1.6.5-py2.7.egg/astroid/scoped_nodes.py", line 2642, in _compute_mro
mro = base._compute_mro(context=context)
File "/home/user/cache-python/eggs/astroid-1.6.5-py2.7.egg/astroid/scoped_nodes.py", line 2655, in _compute_mro
return _c3_merge(unmerged_mro, self, context)
File "/home/user/cache-python/eggs/astroid-1.6.5-py2.7.egg/astroid/scoped_nodes.py", line 65, in _c3_merge
mros=sequences, cls=cls, context=context)
InconsistentMroError: Cannot create a consistent method resolution order for MROs (Tabs, Collection, Resource, LockableItem, EtagSupport, Traversable), (Element, Node), (Collection, Resource, LockableItem, EtagSupport), (Item, Resource, LockableItem, EtagSupport, CopySource, Tabs, Traversable, Element, Node, Owned, Owned, UndoSupport), (FindSupport), (Collection, Item, FindSupport) of class <ClassDef.Folder l.73 at 0x7fa1a07fafd0>.
************* Module test
F: 1, 0: <class 'astroid.exceptions.InconsistentMroError'>: Cannot create a consistent method resolution order for MROs (Tabs, Collection, Resource, LockableItem, EtagSupport, Traversable), (Element, Node), (Collection, Resource, LockableItem, EtagSupport), (Item, Resource, LockableItem, EtagSupport, CopySource, Tabs, Traversable, Element, Node, Owned, Owned, UndoSupport), (FindSupport), (Collection, Item, FindSupport) of class <ClassDef.Folder l.73 at 0x7fa1a07fafd0>. (astroid-error)
Expected behavior
That the pylint worked and indicated some errors in the file.
pylint --version output
No config file found, using default configuration
pylint 1.9.3,
astroid 1.6.5
Python 2.7.14 (default, Nov 13 2017, 14:36:40)
[GCC 5.4.0 20160609]
I've spent some time trying to figure out what's the problem here. The culprit is Folder from OFS.Folder. Unfortunately the only tool capable of getting the MRO of this class is Python itself. I've tried with the standard MRO implementation, I've tried with MRO implementation from astroid and the one from functools, which is used for singledispatch, every single one of them is failing to get the MRO of Folder:
def merge(seqs):
print '\n\nCPL[%s]=%s' % (seqs[0][0],seqs),
res = []; i=0
while 1:
nonemptyseqs=[seq for seq in seqs if seq]
if not nonemptyseqs: return res
i+=1; print '\n',i,'round: candidates...',
for seq in nonemptyseqs: # find merge candidates among seq heads
cand = seq[0]; print ' ',cand,
nothead=[s for s in nonemptyseqs if cand in s[1:]]
if nothead: cand=None #reject candidate
else: break
if not cand: raise Exception("Inconsistent hierarchy")
res.append(cand)
for seq in nonemptyseqs: # remove cand
if seq[0] == cand: del seq[0]
def mro(C):
"Compute the class precedence list (mro) according to C3"
return merge([[C]]+map(mro,C.__bases__)+[list(C.__bases__)])
def _c3_merge(sequences):
"""Merges MROs in *sequences* to a single MRO using the C3 algorithm.
Adapted from http://www.python.org/download/releases/2.3/mro/.
"""
result = []
while True:
sequences = [s for s in sequences if s] # purge empty sequences
if not sequences:
return result
for s1 in sequences: # find merge candidates among seq heads
candidate = s1[0]
for s2 in sequences:
if candidate in s2[1:]:
candidate = None
break # reject the current head, it appears later
else:
break
if candidate is None:
raise RuntimeError("Inconsistent hierarchy")
result.append(candidate)
# remove the chosen candidate
for seq in sequences:
if seq[0] == candidate:
del seq[0]
def _c3_mro(cls, abcs=None):
"""Computes the method resolution order using extended C3 linearization.
If no *abcs* are given, the algorithm works exactly like the built-in C3
linearization used for method resolution.
If given, *abcs* is a list of abstract base classes that should be inserted
into the resulting MRO. Unrelated ABCs are ignored and don't end up in the
result. The algorithm inserts ABCs where their functionality is introduced,
i.e. issubclass(cls, abc) returns True for the class itself but returns
False for all its direct base classes. Implicit ABCs for a given class
(either registered or inferred from the presence of a special method like
__len__) are inserted directly after the last ABC explicitly listed in the
MRO of said class. If two implicit ABCs end up next to each other in the
resulting MRO, their ordering depends on the order of types in *abcs*.
"""
for i, base in enumerate(reversed(cls.__bases__)):
if hasattr(base, '__abstractmethods__'):
boundary = len(cls.__bases__) - i
break # Bases up to the last explicit ABC are considered first.
else:
boundary = 0
abcs = list(abcs) if abcs else []
explicit_bases = list(cls.__bases__[:boundary])
abstract_bases = []
other_bases = list(cls.__bases__[boundary:])
for base in abcs:
if issubclass(cls, base) and not any(
issubclass(b, base) for b in cls.__bases__
):
# If *cls* is the class that introduces behaviour described by
# an ABC *base*, insert said ABC to its MRO.
abstract_bases.append(base)
for base in abstract_bases:
abcs.remove(base)
explicit_c3_mros = [_c3_mro(base, abcs=abcs) for base in explicit_bases]
abstract_c3_mros = [_c3_mro(base, abcs=abcs) for base in abstract_bases]
other_c3_mros = [_c3_mro(base, abcs=abcs) for base in other_bases]
return _c3_merge(
[[cls]] +
explicit_c3_mros + abstract_c3_mros + other_c3_mros +
[explicit_bases] + [abstract_bases] + [other_bases]
)
from OFS.Folder import Folder
# the usual algorithm
mro(Folder) # inconsistent hierarchy
# the one from functools
_c3_merge(Folder) # inconsistent hierarchy
# from astroid
from astroid import extract_node, ClassDef
n = next(extract_node(''''
from OFS.Folder import Folder
Folder
''').infer())
assert isinstance(n, ClassDef)
n.mro() # inconsistent hierarchy
If even the function from functools can't determine the MRO for a live class, I'm not sure how astroid is supposed to determine the MRO from a tree representation of the code.
Going to leave this opened for now, but not sure what's going on here.
Uhm, maybe we can get some help from Zope guys? They may not be familiar with pylint but they may be with some Zope internals. @icemac @tseaver @hannosch
I don't know anything about astroid, but I can suggest at least one case to consider: a bunch of the classes in the OFS.Folder.Folder inheritance hierarchy derive from ExtensionClass.Base, which is a type defined in C: it is possible that the Base type is missing some introspection feature which astroid relies on for builtin / extension types.
Thanks for the insight @tseaver ! Sounds plausible, I'll try to see when I get some time if this is the cause of this error.
@PCManticore I don't know exactly what you need to do here but you couldn't use the python __mro__ attribute?
from OFS.Folder import Folder
print Folder.__mro__
(<class 'OFS.Folder.Folder'>, <class 'OFS.ObjectManager.ObjectManager'>, <class 'OFS.CopySupport.CopyContainer'>, <class 'App.Management.Navigation'>, <class 'App.Management.Tabs'>, <type 'Acquirer'>, <type 'Persistence.Persistent'>, <type 'persistent.Persistent'>, <class 'webdav.Collection.Collection'>, <class 'webdav.Resource.Resource'>, <class webdav.Lockable.LockableItem at 0x7fb4bfbb00b8>, <class webdav.EtagSupport.EtagSupport at 0x7fb4bfe09f58>, <class OFS.Traversable.Traversable at 0x7fb4bf19f390>, <class 'OFS.PropertyManager.PropertyManager'>, <class OFS.ZDOM.ElementWithAttributes at 0x7fb4bf19f668>, <class OFS.ZDOM.Element at 0x7fb4bf19f600>, <class OFS.ZDOM.Node at 0x7fb4bf19f530>, <class 'OFS.role.RoleManager'>, <class 'AccessControl.rolemanager.RoleManager'>, <class AccessControl.PermissionMapping.RoleManager at 0x7fb4bf3062c0>, <class 'OFS.SimpleItem.Item'>, <class 'OFS.CopySupport.CopySource'>, <class 'OFS.owner.Owned'>, <class 'AccessControl.owner.Owned'>, <class 'App.Undo.UndoSupport'>, <class 'OFS.FindSupport.FindSupport'>, <type 'ExtensionClass.Base'>, <type 'object'>)
This looks like the order used by python.
Maybe you can convert this tuple to the format you need.
@idgserpro We don't use the Python objects. Since pylint is a static analysis tool, we have to implement our own MRO computation, which is done statically against the class hierarchy we can infer.
@PCManticore note that __mro__ is a class attribute, not an object attribute. In your attempts to discover the MRO you also use the class attribute:
def mro(C):
"Compute the class precedence list (mro) according to C3"
return merge([[C]]+map(mro,C.__bases__)+[list(C.__bases__)])
Sorry if I'm saying something nonsense.
@idgserpro That's the thing, we don't have access to that C class, instead we use our own representation of C built from the source code.
@PCManticore I updated the description to add the Products.CMFPlone dependency in buildout and upgrade the version of pylint to 1.9.3
@PCManticore the ExtensionClass.Base class referenced by @tseaver isn't the problem here. The astroid simply remove it of the inheritance stack, since it isn't possible to infer its attributes.
I tested another class with more simple inheritance but that inherited from it and it worked.
@PCManticore contrary to what I said in the previous comment, it's just the ExtensionClass.Base class that's causing the problem.
It seems that python changes the order of mro when it encounters a class implemented in C in the inheritance hierarchy. First I made an example without having an ExtensionClass.Base class in the inheritance hierarchy and compares with the python mro:
from astroid import extract_node, ClassDef
from ExtensionClass import Base
print 'Example with class D inheriting from object:\n'
n = next(extract_node("""
from ExtensionClass import Base
class F(object):
pass
class E(object):
pass
class D(object):
pass
class C(F, E):
pass
class B(D, E):
pass
class A(B, C):
pass
A
""").infer())
assert isinstance(n, ClassDef)
print 'astroid and python mro:'
print n.mro()
class F(object):
pass
class E(object):
pass
class D(object):
pass
class C(F, E):
pass
class B(D, E):
pass
class A(B, C):
pass
print A.__mro__
The result:
Example with class D inheriting from object:
astroid and python mro:
[<ClassDef.A l.19 at 0x7ff64d3ca210>, <ClassDef.B l.16 at 0x7ff64d3ca150>, <ClassDef.D l.10 at 0x7ff64d3b8f90>, <ClassDef.C l.13 at 0x7ff64d3ca050>, <ClassDef.F l.4 at 0x7ff64d3b8e10>, <ClassDef.E l.7 at 0x7ff64d3b8e50>, <ClassDef.object l.0 at 0x7ff64d881590>]
(<class '__main__.A'>, <class '__main__.B'>, <class '__main__.D'>, <class '__main__.C'>, <class '__main__.F'>, <class '__main__.E'>, <type 'object'>)
Note that the order is the same.
So I've changed the inheritance of class D from object to Base, both in the astroid part and in the python part. The result was:
Example with class D inheriting from ExtensionClass.Base:
astroid and python mro:
[<ClassDef.A l.19 at 0x7f7f2edec210>, <ClassDef.B l.16 at 0x7f7f2edec150>, <ClassDef.D l.10 at 0x7f7f2ed86f90>, <ClassDef.C l.13 at 0x7f7f2edec050>, <ClassDef.F l.4 at 0x7f7f2ed86e10>, <ClassDef.E l.7 at 0x7f7f2ed86e50>, <ClassDef.object l.0 at 0x7f7f2f24e590>]
(<class '__main__.A'>, <class '__main__.B'>, <class '__main__.D'>, <class '__main__.E'>, <class '__main__.C'>, <class '__main__.F'>, <type 'ExtensionClass.Base'>, <type 'object'>)
Note that in python, class E came before class C.
It seems that when there is a class implemented in C in the inheritance, python puts the class in mro without verifying that it occurs later in the inheritance. In other words, It stops doing this discarding.
In my example, it followed the order:
A -> B -> D -> E
When it got here, it didn't continue checking and see that the E was in the class C inheritance. It already put the E in the mro without checking this.
So to fix this, we have to detect that a class implemented in C is in the inheritance and stop doing the discard. Placing the classes in mro as soon as they are found.
In the case of python in previous comment, when class "D" inherits from Base, mro changed because python considered class "A" as old style. See here and here. For a class to be considered as new style, it must inherit from a basic type of python. But when a C type is found in the inheritance, it seems that python considers the class to be old style and applies the old style algorithm to define mro. That is why the order changes.
astroid doesn't try to determine mro when it detects that the class is old style. See.
When the C3 algorithm, which is used in new style classes, is applied in classes that python considers old style, two things can occur:
-
In classes with simpler inheritance, like the one of the previous comment, the order stays directing of the one of the python.
-
In class with more complex inheritance, like the
plone.dexterity.content.Containerclass of the description, anInconsistentMroErroroccurs.
So the problem in description of the issue is that the astroid could't detect that the class plone.dexterity.content.Container is old style, since it inherits of ExtensionClass.Base.
To fix the problem, we must detect if the class inherits from a type in C, and consider it as old style, so as not to apply the C3 algorithm on it.
@PCManticore do you agree?
I run the code of https://github.com/PyCQA/pylint/issues/2188#issuecomment-423714441 with the class D inheriting from greenlet.greenlet, which is a type implemented in C, and the order of mro hasn't changed. The result was:
Example with class D inheriting from greenlet.greenlet:
[<ClassDef.A l.19 at 0x7f4a26cdb310>, <ClassDef.B l.16 at 0x7f4a26cdb290>, <ClassDef.D l.10 at 0x7f4a26cdb050>, <ClassDef.C l.13 at 0x7f4a26cdb150>, <ClassDef.F l.4 at 0x7f4a26cc9f50>, <ClassDef.E l.7 at 0x7f4a26cc9f90>, <ClassDef.object l.0 at 0x7f4a27191590>]
(<class '__main__.A'>, <class '__main__.B'>, <class '__main__.D'>, <type 'greenlet.greenlet'>, <class '__main__.C'>, <class '__main__.F'>, <class '__main__.E'>, <type 'object'>)
So, the fact that a class inherits from a type in C doesn't change the mro. In the case of ExtensionClass.Base, it is something specific to your implementation that changes the order. This cannot be detected by astroid.
What I think can be done is create a setting to not check the mro of a specific class. It is better that an error be raised. @PCManticore , what is the consequence of a class not having the mro checked?