TCP connection closed while using ragas with fastapi uvicorn

[kaustubh-darekar]

• [x] I have checked the documentation and related resources and couldn't resolve my bug.

Describe the bug I am using ragas for my application where i am calling ragas evaluation through my code and getting error

Ragas version: 0.1.14 Python version: 3.10

Code to Reproduce in score.py

@app.post('/metric')
async def calculate_metric(question=Form(),context=Form(),answer=Form()):
try:
result = await asyncio.to_thread(get_ragas_metrics,question,context,answer)
print("Type of score dict in score :",type(result))
return create_api_response('Success',data=result,message=f"Status set to Reprocess for filename : {result}")
except Exception as e:
job_status = "Failed"
message="Error while calculating evaluation metrics"
error_message = str(e)
logging.exception(f'{error_message}')`
return create_api_response(job_status, message=message, error=error_message)
finally:
gc.collect()

in ragas_eval.py

from ragas.metrics import (
    answer_relevancy,
    faithfulness,
    context_utilization
)
from datasets import Dataset
from ragas import evaluate
def get_ragas_metrics(question,context,answer,model):
try:
dataset = {
'question': [question],
'answer': [answer],
'contexts':[[context]],
}
dataset = Dataset.from_dict(dataset)
logging.info("Dataset created successfully.")

score = evaluate(dataset=dataset,metrics=[faithfulness, answer_relevancy, context_utilization])
score_dict = score.to_pandas()
[['faithfulness','answer_relevancy','context_utilization']].to_dict(orient="index")[0]
logging.info("Evaluation completed successfully.")
print("Type of score : ",type(score_dict))
return score_dict
except Exception as e:
logging.error(f"An error occurred during metrics evaluation: {e}")
return None

Error trace 2024-10-04 06:58:12,210 - Task exception was never retrieved future: <Task finished name='Task-32' coro=<AsyncClient.aclose() done, defined at /opt/conda/envs/myenv310/lib/python3.10/site-packages/httpx/_client.py:2011> exception=RuntimeError('unable to perform operation on <TCPTransport closed=True reading=False 0x76bd38186340>; the handler is closed')> Traceback (most recent call last): File "/opt/conda/envs/myenv310/lib/python3.10/site-packages/httpx/_client.py", line 2018, in aclose await self._transport.aclose() File "/opt/conda/envs/myenv310/lib/python3.10/site-packages/httpx/_transports/default.py", line 385, in aclose await self._pool.aclose() File "/opt/conda/envs/myenv310/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 313, in aclose await self._close_connections(closing_connections) File "/opt/conda/envs/myenv310/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 305, in _close_connections await connection.aclose() File "/opt/conda/envs/myenv310/lib/python3.10/site-packages/httpcore/_async/connection.py", line 171, in aclose await self._connection.aclose() File "/opt/conda/envs/myenv310/lib/python3.10/site-packages/httpcore/_async/http11.py", line 265, in aclose await self._network_stream.aclose() File "/opt/conda/envs/myenv310/lib/python3.10/site-packages/httpcore/_backends/anyio.py", line 54, in aclose await self._stream.aclose() File "/opt/conda/envs/myenv310/lib/python3.10/site-packages/anyio/streams/tls.py", line 202, in aclose await self.transport_stream.aclose() File "/opt/conda/envs/myenv310/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 1187, in aclose self._transport.write_eof() File "uvloop/handles/stream.pyx", line 699, in uvloop.loop.UVStream.write_eof File "uvloop/handles/handle.pyx", line 159, in uvloop.loop.UVHandle._ensure_alive RuntimeError: unable to perform operation on <TCPTransport closed=True reading=False 0x76bd38186340>; the handler is closed

Expected behavior Code should evaluate and gracefully close TCP connection

[dosubot[bot]]

Hello, @kaustubh-darekar! I'm here to assist you with your bug and help you become a contributor. Let's work together to resolve the issue you're facing.

The error you're encountering is related to the asynchronous handling of HTTP connections, specifically with the httpx library and its aclose method. This issue often arises when the connection is closed unexpectedly or prematurely.

Here are a few suggestions to help mitigate this issue:

1. Ensure Proper Cleanup: Make sure that all HTTP clients are properly closed after their use. If you're using an httpx.AsyncClient, ensure that you call await client.aclose() in a finally block or use an asynchronous context manager (async with).

2. Check for Unawaited Tasks: Ensure that all asynchronous operations are awaited. Unawaited tasks can lead to unexpected behavior and resource leaks.

3. Review Connection Pooling: If you're using connection pooling, ensure that the pool is properly managed and closed. This can sometimes lead to issues if connections are not released back to the pool.

4. Increase Timeout Settings: Sometimes, increasing the timeout settings for your HTTP client can help if the issue is related to network latency or slow responses.

5. Use Latest Versions: Ensure that you are using the latest versions of httpx, uvicorn, and any other related libraries, as updates may contain bug fixes for issues like this.

6. Debugging: Add more logging around the areas where HTTP requests are made and closed to better understand the flow and identify where the connection might be closing unexpectedly.

By implementing these practices, you should be able to handle the TCP connection more gracefully and avoid the RuntimeError you're experiencing.

To continue talking to Dosu, mention @dosu.

[anistark]

Please upgrade to latest ragas and use aevaluate().

[github-actions[bot]]

It seems the issue was answered, closing this now.