ragas icon indicating copy to clipboard operation
ragas copied to clipboard

I can't generate Test Data.

Open nandhu-nr opened this issue 1 year ago • 1 comments

----> 1 testset = generator.generate_with_langchain_docs(documents, test_size=10, distributions={simple: 0.5, reasoning: 0.25, multi_context: 0.25},raise_exceptions=False)

2 frames /usr/local/lib/python3.10/dist-packages/ragas/testset/docstore.py in add_nodes(self, nodes, show_progress) 251 results = executor.results() 252 if not results: --> 253 raise ExceptionInRunner() 254 255 for i, n in enumerate(nodes):

ExceptionInRunner: The runner thread which was running the jobs raised an exeception. Read the traceback above to debug it. You can also pass raise_exceptions=False incase you want to show only a warning message instead.

nandhu-nr avatar Oct 04 '24 08:10 nandhu-nr

Hey @nandhu-nr! I'm here to help you with that issue. Let's get to the bottom of it together.

The ExceptionInRunner error occurs when an exception is raised in the executor, specifically when the results from the executor are empty, indicating that the runner thread encountered an issue. To debug this, you should review the traceback provided in the error message. Additionally, since you have set raise_exceptions=False, it should only show a warning message instead of raising the exception, but it seems like the exception is still being raised.

You might want to check if there are any issues with the input documents or the configuration of the generate_with_langchain_docs function. Ensure that the documents parameter is correctly formatted and that all other parameters are set as expected. If the problem persists, consider checking for any updates or patches to the Ragas library that might address this issue.

For similar issues, you can refer to the closed issue Testset generation. ValueError: a cannot be empty unless no samples are taken, which was resolved by a pull request [1].

To continue talking to Dosu, mention @dosu.

dosubot[bot] avatar Oct 04 '24 08:10 dosubot[bot]

imagine using bot to answer this kind of questions

HoiDam avatar Nov 18 '24 09:11 HoiDam