pickling of Exceptions and kw_only

[arnimarj]

Just stumbled across an interesting issue. The kw_only flag seems to break pickling of objects. E.g.:

import attr
import pickle


@attr.s(auto_exc=True, kw_only=False)
class Positional(Exception):
    field: int = attr.ib()


@attr.s(slots=False, auto_exc=True, kw_only=True)
class KwOnly(Exception):
    field: int = attr.ib()


A = Positional(field=1)
B = KwOnly(field=1)

print('A', pickle.loads(pickle.dumps(A)))
print('B', pickle.loads(pickle.dumps(B)))

raises this error

Traceback (most recent call last):
  File "attrs_kwonly.py", line 19, in <module>
    print('B', pickle.loads(pickle.dumps(B)))
TypeError: __init__() takes 1 positional argument but 2 were given

I couldn't find any mention in previous issues, or current documentation about this behaviour.

[Drino]

Hello!

It seems that this bug appears only for inheritors of Exception:

import attr
import pickle


@attr.s(slots=False, auto_exc=True, kw_only=True)
class KwOnly(object):
    field: int = attr.ib()


B = KwOnly(field=1)
print('B', pickle.loads(pickle.dumps(B)))  # B KwOnly(field=1)

I think the issue here is the way Exception is pickled - it is using __reduce__ to get (cls, args, state) tuple, which works incorrectly in this case:

B.__reduce__()  # (__main__.KwOnly, (1,), {'field': 1})
# If we don't use auto_exc=True
B.__reduce__()  # (__main__.KwOnly, (), {'field': 1})
# Note that pickle is still broken, as __init__ is called

I believe it's a bug in auto_exc and I'm not sure how it should be fixed - it seems that __reduce__ (which is default Exception pickling method and is inherited by child class) is not compatible with kw_only-arguments.

You can probably avoid this issue by providing default to all kw_only-arguments and re-implementing own reduce like:

def __reduce__(self): return (self.__class__, (), self.__dict__)

[dragonpaw]

This seems to actually affect when any field on the exception is marked kw_only:

@define(auto_exc=True)
class CantPickleMeSadly(Exception):
    message: None | str = field(kw_only=False, default=None)
    response: None | requests.Response = field(kw_only=True, default=None)

pickle.loads(pickle.dumps(CantPickleMeSadly('foo')))
TypeError: CantPickleMeSadly.__init__() takes from 1 to 2 positional arguments but 3 were given

[hynek]

I guess the solution is as @Drino suggested: implement a def __reduce__(self): return (self.__class__, (), self.__dict__) if any kind of kw_argsis involved.