llama_index
llama_index copied to clipboard
[Bug]: RAGAS integration with llamindex issue
Bug Description
So I want to get the metrics from RAGAS and whenever I dowwnload model locally in my EC2 the metrics gives me the answer biut when I use sagemaker end point as a LLM it thros error.
The error says that it is asyncio error and I need to do nest.apply(). I laready did it but the actual error message is cannot reuse already awaited coroutine
Version
0.10.55
Steps to Reproduce
Here i the code that can be run to priduce the error message
import nest_asyncio
llm = SageMakerLLM(
endpoint_name=<>,
region_name=<>
)
embed_model = HuggingFaceEmbedding(model_name="sentence-transformers/all-mpnet-base-v2", device="cuda")
Settings.llm = llm
Settings.embed_model = embed_model
nest_asyncio.apply()
storage_context = StorageContext.from_defaults(persist_dir=<>)
index = load_index_from_storage(storage_context)
# define evaluator
evaluator = FaithfulnessEvaluator(llm=llm)
# query index
query_engine = index.as_query_engine()
query = "What battles took place in New York City in the American Revolution?"
response = query_engine.query("What battles took place in New York City in the American Revolution?")
eval_result = evaluator.evaluate_response(query=query, response=response)
print(str(eval_result.passing))
Relevant Logs/Tracbacks
No response