PicklingError: Can't pickle <class 'kiwi.data.encoders.wmt_qe_data_encoder.InputFields[PositiveInt]'>: attribute lookup InputFields[PositiveInt] on kiwi.data.encoders.wmt_qe_data_encoder failed

[Blinkblade]

Describe the bug In Windows10,when I try to train_from_file in my Jupyter,I will fail in "Validation sanity check" and the erro is PicklingError:"Can't pickle <class 'kiwi.data.encoders.wmt_qe_data_encoder.InputFields[PositiveInt]'>: attribute lookup InputFields[PositiveInt] on kiwi.data.encoders.wmt_qe_data_encoder failed". But when I run it in Linux with the same files,it will be all right with no error.

To Reproduce Steps to reproduce the behavior: 1.My code in Jupyter: from kiwi.lib.train import train_from_file import pathlib congfig = pathlib.Path("config/bert.yaml") run_info = train_from_file(congfig)

2.my bert.yaml: bert.yaml.txt

3.Then it will load model and data,until it begin to "Validation sanity check",there will be this PicklingError.

Expected behavior PicklingError: Can't pickle <class 'kiwi.data.encoders.wmt_qe_data_encoder.InputFields[PositiveInt]'>: attribute lookup InputFields[PositiveInt] on kiwi.data.encoders.wmt_qe_data_encoder failed

Screenshots

Environment (please complete the following information):

• OS: Windows10
• OpenKiwi version:2.0.0
• Python version:3.8.8

I have no idea about this Error,Could someone help me？Thanks.

[jlx339]

I encountered the same error, please help.

[dmar1n]

Hi!

I also encountered a 'PicklingError' when trying to train in a multi-GPU machine. If this is your case, you can maybe set one single device with CUDA_VISIBLE_DEVICES. That worked for me at least.