Documents appears to be too short (ie 100 tokens or less). Please provide longer documents.

[ananthanarayanan431]

[ ] I have checked the documentation and related resources and couldn't resolve my bug.

Describe the bug A clear and concise description of what the bug is. ValueError Traceback (most recent call last) Cell In[38], line 19 12 generator = TestsetGenerator.from_langchain( 13 llm=generator_llm, 14 embedding_model=generator_embeddings, 15 ) 17 query_distribution = default_query_distribution(generator_llm) ---> 19 testset = generator.generate_with_langchain_docs( 20 documents=chunks, 21 testset_size=10, 22 query_distribution=query_distribution, 23 )

File ~/Desktop/project/.venv/lib/python3.12/site-packages/ragas/testset/synthesizers/generate.py:164, in TestsetGenerator.generate_with_langchain_docs(self, documents, testset_size, transforms, transforms_llm, transforms_embedding_model, query_distribution, run_config, callbacks, with_debugging_logs, raise_exceptions) 159 raise ValueError( 160 """An embedding client was not provided. Provide an embedding through the transforms_embedding_model parameter. Alternatively you can provide your own transforms through the transforms parameter.""" 161 ) 163 if not transforms: --> 164 transforms = default_transforms( 165 documents=list(documents), 166 llm=transforms_llm or self.llm, 167 embedding_model=transforms_embedding_model or self.embedding_model, 168 ) 170 # convert the documents to Ragas nodes ... 161 "Documents appears to be too short (ie 100 tokens or less). Please provide longer documents." 162 ) 164 return transforms

ValueError: Documents appears to be too short (ie 100 tokens or less). Please provide longer documents.

Ragas version: 0.2.15 Python version: 3.12

Code to Reproduce

from ragas.llms.base import LangchainLLMWrapper from ragas.embeddings.base import LangchainEmbeddingsWrapper from langchain_openai import ChatOpenAI, OpenAIEmbeddings

from ragas.testset import TestsetGenerator from ragas.testset.synthesizers import default_query_distribution

generator_llm = LangchainLLMWrapper(langchain_llm=ChatOpenAI(model="gpt-4o-mini")) generator_embeddings = LangchainEmbeddingsWrapper(embeddings=OpenAIEmbeddings(model="text-embedding-3-small"))

generator = TestsetGenerator.from_langchain( llm=generator_llm, embedding_model=generator_embeddings, )

query_distribution = default_query_distribution(generator_llm)

testset = generator.generate_with_langchain_docs( documents=chunks, testset_size=10, query_distribution=query_distribution, )

Error trace ValueError

Expected behavior Creation of test dataset

Additional context Add any other context about the problem here.