Cython-Python pickling

[asvetlov]

Pickled pure Python MultiDicts should be unpickled as Cythonized if unpicker is executed on machine with Cython support. And vise versa.

[gyermolenko]

There are protocols 0..4 (inclusive). Maybe I could bump upper range in these places? gen_pickles.py conftest.py

ps after mentioned +1 number of tests turn from 36 to 44. And they pass

[gyermolenko]

Regarding the issue itself.. Could you share some thoughts about desirable implementation? Does it involve custom logic in __getnewargs__, __reduce__ and a bunch of other magic methods depending on USE_CYTHON variable?

[asvetlov]

My idea is to look on CPython sources. Unfortunately, that's it for now

[gyermolenko]

Here is what I got playing around with special methods related to pickle protocol:

import pickle

USE_CYTHON = True
# USE_CYTHON = False

class A:
    def __getnewargs_ex__(self):
        # print('in __getnewargs_ex__ of A')
        if USE_CYTHON:
            print('selecting path B')
            return ((B,), {})
        else:
            print('selecting path A')
            return ((A,), {})

    def __new__(cls, *args, **kwargs):
        # print('In A __new__, *args, **kwargs: ', *args, **kwargs)
        if args:
            return args[0]()
        else:
            return super().__new__(cls)

class B: pass

if __name__ == '__main__':
    obj = A()
    pd = pickle.dumps(obj)
    pl = pickle.loads(pd)
    #                           USE_CYTHON = True          USE_CYTHON = False
    print(pl.__class__)       # <class '__main__.B'>  or   <class '__main__.A'>