.reload for modules with subclasses that call super leads to TypeError

[HumorBaby]

When a module that defines subclasses that call super, a reload will lead to an exception because TypeError: super(type, obj): obj must be an instance or subtype of type. Apparently in Python 2 (not sure about 3), when reloading modules, isinstance checks for running super will fail due to internal memory changes.

See: http://thomas-cokelaer.info/blog/2011/09/382/ https://thingspython.wordpress.com/2010/09/27/another-super-wrinkle-raising-typeerror/

[HumorBaby]

Just don't use super. Do Exception.__init__(self, msg) instead of super(MyException, self).__init__(msg).

[Exirel]

I've been reading about this, and there is no good solution here in Python 2, and there is an excellent one with Python 3. Here is a Python shell session:

>>> import sopelunker
>>> import importlib
>>> 
>>> 
>>> c = sopelunker.Child()
>>> c.sopelunking()
making PR #3400
>>> c.py2compat()
making PR with py2 compat: #9007
>>> 
>>> 
>>> importlib.reload(sopelunker)
<module 'sopelunker' from '/home/sopel/workspace/unking/sopelunker.py'>
>>> 
>>> 
>>> c.sopelunking()
making PR #9162
>>> c.py2compat()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/sopel/workspace/unking/sopelunker.py", line 16, in py2compat
    super(Child, self).py2compat(random.randint(1000, 9999))
TypeError: super(type, obj): obj must be an instance or subtype of type

And here is the Python code itself:

import random

class Mother(object):
    def sopelunking(self, number):
        print('making PR #%d' % number)

    def py2compat(self, number):
        print('making PR with py2 compat: #%d' % number)


class Child(Mother):
    def sopelunking(self):
        super().sopelunking(random.randint(1000, 9999))

    def py2compat(self):
        super(Child, self).py2compat(random.randint(1000, 9999))

With Python 3, you can use super() directly, unlike Python 2, where you'll get an error: TypeError: super() takes at least 1 argument (0 given).

So that's not something that can be fixed in Python 2, because that's how reload works, and there is nothing you can do about it - at least, nothing you can do while keeping a clean code, ready for Python 3.

[HumorBaby]

Yep, I have found the same: the simplest Python 2 solution would require a change in the module iteslf. I had really opened this issue as a note for others that may have been having a similar problem with their own third-party modules since none of the modules that come with Sopel would suffer from this. For Py2, I don't see a way to have Sopel handle this issue. @dgw, might be worth updating milestones and labels at some point based on this.

[dgw]

@Exirel Since this can't be fixed in Python 2, let's push this to the 8.0.0 milestone? The draft migration guide I have for Sopel 7 explicitly says we'll drop all EOL Python versions in that release, including 2.7, which sounds like a stellar plan. (No more testing on 3.3!)

[Exirel]

We don't have many options. :-/

[dgw]

@HumorBaby @Exirel Do we need to do anything about this, or is it fixed automatically by dropping support for py2 in #2062?

[Exirel]

I wish, but I'm absolutely not sure. This need to be tested.