archivematica
archivematica copied to clipboard
Use JSON as the wire format in Gearman jobs
This pull request replaces all uses of pickle
with json
. The use of pickle
was introduced back in 2011 when MCP was integrated with Gearman due to its ability to round-trip Python objects, at the cost of lower security. However, apart from datetime.datetime
and uuid.UUID
, we are not encoding any complex structures that cannot be easily represented in JSON.
Please notice that the use of orjson
was an afterthought and that's why it was submitted as an extra commit. orjson
is significantly faster compared to the standard library json
module, being faster than pickle protocol=0 that we were using previously. It also supports native serialization of datetime and UUID objects.
Fixes https://github.com/archivematica/Issues/issues/1673.