执行：python index.py zh ../data/dureader.para test_index 报UnicodeDecodeError错

[jamyriver]

执行：python index.py zh ../data/dureader.para test_index 报错： ****\anaconda3\envs\my_paddlenlp\lib\site-packages\pkg_resources_init_.py:121: DeprecationWarning: pkg_resources is deprecated as an API warnings.warn("pkg_resources is deprecated as an API", DeprecationWarning) ****\anaconda3\envs\my_paddlenlp\lib\site-packages\pkg_resources_init_.py:2870: DeprecationWarning: Deprecated call to pkg_resources.declare_namespace('mpl_toolkits'). Implementing implicit namespace packages (as specified in PEP 420) is preferred to pkg_resources.declare_namespace. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages declare_namespace(pkg) ****\anaconda3\envs\my_paddlenlp\lib\site-packages\pkg_resources_init_.py:2870: DeprecationWarning: Deprecated call to pkg_resources.declare_namespace('google'). Implementing implicit namespace packages (as specified in PEP 420) is preferred to pkg_resources.declare_namespace. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages declare_namespace(pkg) Traceback (most recent call last): File "index.py", line 41, in for line in open(data_file): UnicodeDecodeError: 'gbk' codec can't decode byte 0x80 in position 4: illegal multibyte sequence

[jamyriver]

自己解决了， 把 index.py 的 41 行改为：for line in open(data_file,"r", encoding="UTF-8", errors="ignore"):