xnmt icon indicating copy to clipboard operation
xnmt copied to clipboard

Sentencepiece serialization error

Open pmichel31415 opened this issue 7 years ago • 1 comments

I thought I solved it in #351but there is still a

TypeError: can't pickle SwigPyObject objects

When preprocessing with sentencepiece. I'm guessing it's because the sentencepiece processor can't be pickled but I can't find a way to exclude an attribute from a class during serialization. @msperber do you have any ideas?

pmichel31415 avatar Apr 24 '18 23:04 pmichel31415

Hm, I can't spot the problem from looking at the code. Normally, only the things passed as __init__() arguments will get serialized, and if one of the given arguments is from a different library (and therefore does not inherit from xnmt.persistence.Serializable) one would have to create a wrapper (as was done with the classes in optimizer.py). If this hint doesn't help, could you send a config file so I can reproduce this?

msperber avatar Apr 25 '18 05:04 msperber