lightKG icon indicating copy to clipboard operation
lightKG copied to clipboard

KRL的数据读入报错 examples = [make_example(line, fields) for line in reader]报错,请问怎么解决

Open XuanYang1991 opened this issue 6 years ago • 4 comments

_____________________ ERROR collecting test_krl_transe.py _____________________ test_krl_transe.py:26: in krl.train(train_path, model_type=model_type, dev_path=train_path, save_path='./krl_{}_saves'.format(model_type)) ..........\Anaconda3\envs\python367\lib\site-packages\lightkg\krl\module.py:29: in train train_dataset = rl_tool.get_dataset(train_path) ..........\Anaconda3\envs\python367\lib\site-packages\lightkg\krl\tool.py:31: in get_dataset rl_dataset = TabularDataset(path, format=file_type, fields=fields, skip_header=skip_header) ..........\Anaconda3\envs\python367\lib\site-packages\torchtext\data\dataset.py:271: in init examples = [make_example(line, fields) for line in reader] ..........\Anaconda3\envs\python367\lib\site-packages\torchtext\data\dataset.py:271: in examples = [make_example(line, fields) for line in reader] ..........\Anaconda3\envs\python367\lib\site-packages\torchtext\utils.py:130: in unicode_csv_reader csv.field_size_limit(sys.maxsize) E OverflowError: Python int too large to convert to C long ------------------------------- Captured stderr ------------------------------- [2019-11-13 16:58:36] [INFO] [MainThread] [tool.py:30] loading dataset from C:\Users\Administrator\work\jupyter\study\Untitled Folder\kre\train.sample.csv !!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!! =========================== 1 error in 3.36 seconds =========================== Process finished with exit code 0 我的数据和你的一样

科学,包涵,自然、社会、思维等领域 科学,外文名,science 科学,拼音,kē xué 科学,中文名,科学 科学,解释,发现、积累的真理的运用与实践 语法学,外文名,syntactics 语法学,中文名,语法学 物理宇宙学,对象,大尺度结构和宇宙形成 物理宇宙学,时间,二十世纪 物理宇宙学,所属,天体物理学 物理宇宙学,中文名,物理宇宙学 曹禺,出生地,天津 曹禺,毕业院校,清华大学 曹禺,逝世日期,1996年(丙子年)12月13日 曹禺,中文名,万家宝 曹禺,国籍,中国

XuanYang1991 avatar Nov 13 '19 08:11 XuanYang1991

你的文件是什么格式的,是utf-8吗

smilelight avatar Nov 13 '19 10:11 smilelight

是的,utf-8,gbk2312等尝试过

XuanYang1991 avatar Nov 14 '19 01:11 XuanYang1991

请问问题解决了吗?我跟你一样的错误

May-Sunshine avatar Dec 26 '19 08:12 May-Sunshine

@May-Sunshine 看这报的错误,应该不是框架的问题吧?你可以网上搜一搜,看看能不能找到类似的解决方案。

smilelight avatar Dec 26 '19 08:12 smilelight