toml icon indicating copy to clipboard operation
toml copied to clipboard

TOML with inline tables is not pickleable

Open moqmar opened this issue 4 years ago • 3 comments

Is there a reason why get_empty_inline_table uses a local class? It prevents the result from being pickled:

>>> pickle.dumps(toml.loads("test = { fail = true }"))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: Can't pickle local object 'TomlDecoder.get_empty_inline_table.<locals>.DynamicInlineTableDict'

moqmar avatar May 17 '21 21:05 moqmar

Workaround:

>>> class PickleableTomlDecoder(toml.TomlDecoder):
...     def get_empty_inline_table(self):
...         return self.get_empty_table()
... pickle.dumps(toml.loads("test = { fail = true }", decoder=PickleableTomlDecoder()))
b'\x80\x04\x95\x16\x00\x00\x00\x00\x00\x00\x00}\x94\x8c\x04test\x94}\x94\x8c\x04fail\x94\x88ss.'

moqmar avatar May 17 '21 21:05 moqmar

I noticed this too. When using multiprocessing, this is quite disruptive. @moqmar, I found a workaround in the TOML file (at least in my case). It seems like the issue occurs when assigning a dictionary (.e.g. x={a=1, b=2}). However, if the dictionary is broken into

[my.stuff.x]
a = 1
b = 2

It becomes pick-able. I hope this helps.

amor71 avatar Jul 20 '21 12:07 amor71

you can also use json load to avoid this bug:

 json.loads(json.dumps(toml.load("xxxx.toml")))

bulletRush avatar Nov 10 '22 02:11 bulletRush