pytest
pytest copied to clipboard
pytest-dev
Can't unpickle container class while running in pytest
a detailed description of the bug or problem you are having
Pickling a class that serves as just a container for a custom enum results in the inability to unpickle that class when run through pytest. The error message:
$ pytest
======================================================================== test session starts ========================================================================
platform win32 -- Python 3.7.9, pytest-7.1.3, pluggy-1.0.0
rootdir: C:\Users\michael.sendker\enum_test
collected 1 item
test_pickling.py F [100%]
============================================================================= FAILURES ==============================================================================
__________________________________________________________________________ test_read_enums __________________________________________________________________________
def test_read_enums():
with open("pickles.json", 'rt') as f_in:
d = json.load(f_in)
for key, val in d.items():
> loaded_container = pickle.loads(b64.decodebytes(val.encode()))
E AttributeError: Can't get attribute 'ContainerClass' on <module '__main__' from 'C:\\Users\\michael.sendker\\enum_test\\venv\\Scripts\\pytest.exe\\__main_
_.py'>
test_pickling.py:31: AttributeError
====================================================================== short test summary info ======================================================================
FAILED test_pickling.py::test_read_enums - AttributeError: Can't get attribute 'ContainerClass' on <module '__main__' from 'C:\\Users\\michael.sendker\\enum_test\\...
========
Meanwhile, running the same functions through python instead of pytest, the test passes.
To run: first run the code with the regular python interpreter, then run with pytest and the test will error.
output of pip list from the virtual environment you are using
Package Version
------------------ -------
attrs 22.1.0
colorama 0.4.5
importlib-metadata 4.12.0
iniconfig 1.1.1
packaging 21.3
pip 21.3.1
pluggy 1.0.0
py 1.11.0
pyparsing 3.0.9
pytest 7.1.3
setuptools 60.2.0
tomli 2.0.1
typing_extensions 4.3.0
wheel 0.37.1
zipp 3.8.1
pytest and operating system versions
Windows 10 Pro 21H1 Build 19043.2006
minimal example if possible
import pickle
import json
import base64 as b64
class ContainerClass:
def __init__(self, value):
self.value = value
def __repr__(self):
return str(self.value)
def construct_enum_pickles():
d = {}
for value in range(10):
obj = ContainerClass(value)
d[value] = b64.encodebytes(pickle.dumps(obj)).decode()
with open("pickles.json", 'wt') as f_out:
json.dump(d, f_out)
def test_read_enums():
with open("pickles.json", 'rt') as f_in:
d = json.load(f_in)
for key, val in d.items():
loaded_container = pickle.loads(b64.decodebytes(val.encode()))
print(loaded_container.value)
assert loaded_container.value == int(key)
print("It worked.")
if __name__ == '__main__':
test_read_enums()
this is a user-error about code locations, and the reason why its recommended to deploy a console_script entrypoint
if you run the module with pytohn, its named main, if you import it its named as the file
Where is that recommended? Do you have any documentation on this? I thought there was a chance this was user error but its such a weird footgun I need to get around. The only thing about console_scripts I can find is in regards to setup.py and I'm not packaging the script or the testing suite I discovered this issue in.
This isn't a pytest-specific issue; it'll apply to any way of running your code that isn't directly executing this script. See e.g. this stackoverflow question for more details.
