swift
swift copied to clipboard
modelscope
get dataset error
Describe the bug What the bug is, and how to reproduce, better with screenshots(描述bug以及复现过程，最好有截图) 用最新代码，下面获取数据集会错误
from swift.llm.utils import get_dataset
get_dataset("hh-rlhf-cn")
Your hardware and system info Write your system info like CUDA version/system/GPU/torch version here(在这里给出硬件信息和系统信息，如CUDA版本，系统，GPU型号和torch版本等)
Additional context Add any other context about the problem here(在这里补充其他信息)
什么报错哇
SyntaxError: invalid character '、' (U+3001)
row['context'] = ast.literal_eval(row['context'])
File "/opt/conda/lib/python3.10/ast.py", line 64, in literal_eval
node_or_string = parse(node_or_string.lstrip(" \t"), mode='eval')
File "/opt/conda/lib/python3.10/ast.py", line 50, in parse
return compile(source, filename, mode, flags,
File "
解析为 json 的时候，某些数据会报错
SyntaxError: invalid character '、' (U+3001) row['context'] = ast.literal_eval(row['context']) File "/opt/conda/lib/python3.10/ast.py", line 64, in literal_eval node_or_string = parse(node_or_string.lstrip(" \t"), mode='eval') File "/opt/conda/lib/python3.10/ast.py", line 50, in parse return compile(source, filename, mode, flags, File "", line 1
解析为 json 的时候，某些数据会报错
swift版本是什么？
SyntaxError: invalid character '、' (U+3001) row['context'] = ast.literal_eval(row['context']) File "/opt/conda/lib/python3.10/ast.py", line 64, in literal_eval node_or_string = parse(node_or_string.lstrip(" \t"), mode='eval') File "/opt/conda/lib/python3.10/ast.py", line 50, in parse return compile(source, filename, mode, flags, File "", line 1 解析为 json 的时候，某些数据会报错
swift版本是什么？
从当前源码安装的