ipdb.set_trace() prevents Pickling valid top-level function

[bboerner]

Use of ipdb.set_trace() prevents Pickling valid top-level function which precludes use of Python modules which rely on Pickling e.g. multiprocessing. The following reproduces on two Ubuntu boxes - 14.04 LTS and 16.04 LTS.

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import pickle
import ipdb

def f(x):
    return x*x

if __name__ == '__main__':

    try:
        x = pickle.dumps(f)
        print('1st pickle: %s' % x)
    except Exception, e:
        print('exception: %s' % e)

    ipdb.set_trace()

    try:
        x = pickle.dumps(f)
        print('2nd pickle: %s' % x)
    except Exception, e:
        print('exception: %s' % e)

1st pickle: c__main__
f
p0
.
> /home/sandbox/ipdb_pickle_bug.py(20)<module>()
     19 
---> 20     try:
     21         x = pickle.dumps(f)

ipdb> c
exception: Can't pickle <function f at 0x7fe575328e60>: it's not found as __main__.f

[whenov]

This bug costs me quite a bit of time.

[nonagonal]

I just ran into this too. I think it's just the import of ipdb which prevents pickle from pickling any functions defined in main. See this IPython bug (still open unfortunately) for more details: https://github.com/ipython/ipython/issues/1252

[gotcha]

In particular, read

Generally speaking, pickling things that are defined in the main module (functions, classes, instances of said classes, things that reference any of the above) is simply not going to work robustly. You can never be sure that whatever is unpickling is going to have your module as main. The use of Tracer() just made it fail sooner rather than later.

[nonagonal]

That's reasonable I suppose, but now ipdb is preventing a pretty simple use case: a single file run as a script which wants to use multiprocessing.Pool (which requires the pickling of a function).

[gotcha]

As far as I understand, the issue comes from IPython. If someone finds a workaround, PR welcome

[nonagonal]

Well we could do what I'm doing inside ipdb itself: we could save sys.modules['__main__'] before importing anything from IPython, then restore it. I think this will have some effect on IPython though so it's probably not a great idea!