ragas icon indicating copy to clipboard operation
ragas copied to clipboard

ValidationError: 1 validation error for Prompt

Open beatG123 opened this issue 5 months ago • 1 comments

[ ] I have checked the documentation and related resources and couldn't resolve my bug.

Describe the bug When I try to use generator.adapt(language='chinese'), I still encounter errors related to the Prompt. I have tried multiple methods but still cannot resolve the issue.

Ragas version:0.1.18 Python version:3.10.9

Code to Reproduce from utils import * from langchain_community.document_loaders import PyMuPDFLoader

1.读取原始数据

PROJECT_PATH = os.path.dirname(file) data_path = os.path.join(PROJECT_PATH, 'data/eval_document_data/') pdf_files = [f for f in os.listdir(data_path) if f.endswith('.pdf')]

创建一个空列表来存储文档

documents = []

遍历每个 PDF 文件,并使用 PyMuPDFLoader 加载它们

for pdf_file in pdf_files: full_pdf_path = os.path.join(data_path, pdf_file) loader = PyMuPDFLoader(full_pdf_path) documents.extend(loader.load())

if len(documents) != 0: print("数据加载完成。")

print(documents)

2.数据生成

from ragas.testset.generator import TestsetGenerator from ragas.testset.evolutions import simple, reasoning, multi_context

generator with openai models

generator_llm = ChatOpenAI(model="gpt-3.5-turbo-16k") critic_llm = ChatOpenAI(model="gpt-3.5-turbo-16k") embeddings = get_embeddings_model()

generator = TestsetGenerator.from_langchain( generator_llm, critic_llm, embeddings )

设置语言类型

language = 'chinese' generator.adapt(language=language,evolutions=[simple,reasoning,multi_context], cache_dir="a_path") generator.save(evolutions=[simple, reasoning, multi_context], cache_dir="a_path")

testset = generator.generate_with_langchain_docs(documents, test_size=10, distributions={simple: 0.5, reasoning: 0.25, multi_context: 0.25}) print(testset.to_pandas())

Error trace raise validation_error pydantic.v1.error_wrappers.ValidationError: 1 validation error for Prompt root output in example 1 is not in valid json format: Expecting value: line 1 column 1 (char 0) (type=value_error)

Expected behavior A clear and concise description of what you expected to happen.

Additional context Add any other context about the problem here.

beatG123 avatar Sep 14 '24 13:09 beatG123