check50 icon indicating copy to clipboard operation
check50 copied to clipboard

PicklingError when importing cs50 library in checks

Open kzidane opened this issue 5 years ago • 5 comments

Traceback (most recent call last):
  File "/usr/local/bin/check50", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/dist-packages/check50/__main__.py", line 379, in main
    check_results = check_runner.run(args.target)
  File "/usr/local/lib/python3.7/dist-packages/check50/runner.py", line 201, in run
    result, state = future.result()
  File "/usr/lib/python3.7/concurrent/futures/_base.py", line 428, in result
    return self.__get_result()
  File "/usr/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/usr/lib/python3.7/multiprocessing/queues.py", line 236, in _feed
    obj = _ForkingPickler.dumps(obj)
  File "/usr/lib/python3.7/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function <lambda> at 0x7ff58294add0>: attribute lookup <lambda> on cs50.cs50 failed

kzidane avatar Oct 21 '20 15:10 kzidane

Do you have the relevant slug by any chance?

Jelleas avatar Oct 21 '20 15:10 Jelleas

cs50/problems/2020/fall/movies

kzidane avatar Oct 21 '20 16:10 kzidane

Seems to be fixed in the online version of check50, with -l this error still exists.

MarijnDoeve avatar Dec 17 '20 16:12 MarijnDoeve

This happens because the cs50 library sets the sys.excepthook itself unconditionally to a lambda function. That famously are not pickle-able, because where'd be the fun in that.

As it happens, check50 now treats sys.excepthook as something special, because from Python 3.8 on OS X check50 is forced to support the spawn method for multiprocessing. Long story short, this line: https://github.com/cs50/check50/blob/03d5bb037c9323b8ab9d45cdd4d54613a2f8d924/check50/runner.py#L333 now causes check50 to copy w/e excepthook is set to each check process.

In part this problem stems from the awkwardness of a library changing sys.excepthook. No one expects tracebacks to change thanks to this import:

from cs50 import SQL

I think the only approach here is to accept that sys.excepthook can be set externally, either by the checks file, or a library such as python-cs50. If that happens then either under spawn pass nothing, because that imported library should be imported again and set sys.excepthook again. Or under fork don't set sys.excepthook.

Until then, either move from cs50 import SQL from the top of the test file to each test/function where it's needed. Moving that import to line 119 fixes movies for instance.

Jelleas avatar Dec 17 '20 17:12 Jelleas

https://github.com/cs50/check50/tree/unpickleable_attributes

Jelleas avatar Dec 17 '20 17:12 Jelleas