ragas
ragas copied to clipboard
ValidationError: 1 validation error for Prompt
[ ] I have checked the documentation and related resources and couldn't resolve my bug.
Describe the bug When I try to use generator.adapt(language='chinese'), I still encounter errors related to the Prompt. I have tried multiple methods but still cannot resolve the issue.
Ragas version:0.1.18 Python version:3.10.9
Code to Reproduce from utils import * from langchain_community.document_loaders import PyMuPDFLoader
1.读取原始数据
PROJECT_PATH = os.path.dirname(file) data_path = os.path.join(PROJECT_PATH, 'data/eval_document_data/') pdf_files = [f for f in os.listdir(data_path) if f.endswith('.pdf')]
创建一个空列表来存储文档
documents = []
遍历每个 PDF 文件,并使用 PyMuPDFLoader 加载它们
for pdf_file in pdf_files: full_pdf_path = os.path.join(data_path, pdf_file) loader = PyMuPDFLoader(full_pdf_path) documents.extend(loader.load())
if len(documents) != 0: print("数据加载完成。")
print(documents)
2.数据生成
from ragas.testset.generator import TestsetGenerator from ragas.testset.evolutions import simple, reasoning, multi_context
generator with openai models
generator_llm = ChatOpenAI(model="gpt-3.5-turbo-16k") critic_llm = ChatOpenAI(model="gpt-3.5-turbo-16k") embeddings = get_embeddings_model()
generator = TestsetGenerator.from_langchain( generator_llm, critic_llm, embeddings )
设置语言类型
language = 'chinese' generator.adapt(language=language,evolutions=[simple,reasoning,multi_context], cache_dir="a_path") generator.save(evolutions=[simple, reasoning, multi_context], cache_dir="a_path")
testset = generator.generate_with_langchain_docs(documents, test_size=10, distributions={simple: 0.5, reasoning: 0.25, multi_context: 0.25}) print(testset.to_pandas())
Error trace raise validation_error pydantic.v1.error_wrappers.ValidationError: 1 validation error for Prompt root output in example 1 is not in valid json format: Expecting value: line 1 column 1 (char 0) (type=value_error)
Expected behavior A clear and concise description of what you expected to happen.
Additional context Add any other context about the problem here.