Neptune - awswrangler.exceptions.QueryFailed: Connection was already closed

[kukushking]

Describe the bug

Opening on others behalf

Traceback (most recent call last):
File "/opt/python/awswrangler/neptune/client.py", line 190, in _execute_gremlin
results = future_results.result()
File "/var/lang/lib/python3.9/concurrent/futures/_base.py", line 446, in result
return self.__get_result()
File "/var/lang/lib/python3.9/concurrent/futures/_base.py", line 391, in __get_result
raise self._exception
File "/opt/python/gremlin_python/driver/resultset.py", line 90, in cb
f.result()
File "/var/lang/lib/python3.9/concurrent/futures/_base.py", line 439, in result
return self.__get_result()
File "/var/lang/lib/python3.9/concurrent/futures/_base.py", line 391, in __get_result
raise self._exception
File "/var/lang/lib/python3.9/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/opt/python/gremlin_python/driver/connection.py", line 82, in _receive
data = self._transport.read()
File "/opt/python/gremlin_python/driver/aiohttp/transport.py", line 108, in read
raise RuntimeError("Connection was already closed.")
RuntimeError: Connection was already closed.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/var/task/index.py", line 153, in do_graph_query
df = query_node_without_filters(node_id)
File "/var/task/index.py", line 86, in query_node_without_filters
df = wr.neptune.execute_gremlin(neptune_client, query)
File "/opt/python/awswrangler/neptune/neptune.py", line 62, in execute_gremlin
results = client.read_gremlin(query)
File "/opt/python/awswrangler/neptune/client.py", line 167, in read_gremlin
return self._execute_gremlin(query, headers)
File "/opt/python/awswrangler/neptune/client.py", line 197, in _execute_gremlin
raise exceptions.QueryFailed(e)
awswrangler.exceptions.QueryFailed: Connection was already closed.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/python/aws_lambda_powertools/event_handler/api_gateway.py", line 611, in _call_route
return ResponseBuilder(self._to_response(route.func(**args)), route)
File "/opt/python/aws_lambda_powertools/tracing/tracer.py", line 632, in decorate
response = method(*args, **kwargs)
File "/var/task/index.py", line 176, in do_graph_query
raise InternalServerError(be)
aws_lambda_powertools.event_handler.exceptions.InternalServerError: Connection was already closed.

How to Reproduce

neptune_client = wr.neptune.connect(neptune_endpoint, neptune_port, iam_enabled=iam_enabled)
wr.neptune.execute_gremlin(neptune_client, query)

Expected behavior

No response

Your project

No response

Screenshots

No response

OS

Unix

Python version

3.9

AWS DataWrangler version

2.15.1

Additional context

Running in a Lambda env, client created outside handler

[kukushking]

Looks like everything is ok with the sessions as I can't reproduce this in my env, however what might be affecting the connections is clusterQueryTimeoutInMs setting on the Neptune cluster. If the query runs longer than that, the connection will be closed.

[github-actions[bot]]

Marking this issue as stale due to inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 7 days it will automatically be closed.