DeepPavlov
DeepPavlov copied to clipboard
DS_Store exception
When I use dataset_format txt, the training fails because of the DS_Store system file, maybe it would be good to add exceptions for all operating systems. ` rika@MacBook-Pro adp % python -m deeppavlov train -d train.json 2019-11-26 20:25:46.276 INFO in 'deeppavlov.dataset_readers.odqa_reader'['odqa_reader'] at line 57: Reading files... 2019-11-26 20:25:46.277 INFO in 'deeppavlov.dataset_readers.odqa_reader'['odqa_reader'] at line 134: Building the database... 28%|███████████████████████▏ | 84/301 [00:00<00:00, 3976.14it/s] multiprocessing.pool.RemoteTraceback:
""" Traceback (most recent call last): File "/Users/rika/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/pool.py", line 121, in worker result = (True, func(*args, **kwds)) File "/Users/rika/.pyenv/versions/3.7.0/lib/python3.7/site-packages/deeppavlov/dataset_readers/odqa_reader.py", line 181, in _get_file_contents text = fin.read() File "/Users/rika/.pyenv/versions/3.7.0/lib/python3.7/codecs.py", line 322, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe8 in position 10: invalid continuation byte """ `