anonlink-entity-service icon indicating copy to clipboard operation
anonlink-entity-service copied to clipboard

Socket timeout in client with modest parameters

Open unzvfu opened this issue 6 years ago • 1 comments

Got the following timeout while investigating #125.

$ docker run -i \
       -e ENTITY_SERVICE_URL=http://localhost:${port}/api/v1 \
       -e ENTITY_SERVICE_PERMUTATION=1 \
       -e ENTITY_SERVICE_TEST_SIZE=8000 \
       --net=host \
       quay.io/n1analytics/entity-app python - < backend/test_service.py
04:27:03.765                   n1       INFO - Testing Entity Service at URL: http://localhost:32823/api/v1
04:27:03.765                   n1       INFO - Entity Service URL: http://localhost:32823/api/v1
04:27:05.768                   n1       INFO - Carrying out e2e test
04:27:05.768                   n1       INFO - ---> Number of entities: 8000
04:27:05.768                   n1       INFO - Generating local PII data
04:27:05.960                   n1       INFO - Locally hashing party A identity data to create bloom filters
04:27:11.474                   n1       INFO - Locally hashing party B identity data to create bloom filters
04:27:16.972                   n1       INFO - Serialising bloom filters
04:27:23.462                   n1       INFO - Data generation complete
04:27:23.468                   n1       INFO - Data generation took 11.197 seconds
04:27:23.469       n1.mappingtest       INFO - Starting mapping test with dataset size 8000
04:27:30.883       n1.mappingtest       INFO - Creating a new mapping
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 387, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 383, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/lib/python3.6/http/client.py", line 1331, in getresponse
    response.begin()
  File "/usr/local/lib/python3.6/http/client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "/usr/local/lib/python3.6/http/client.py", line 258, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/local/lib/python3.6/socket.py", line 586, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:
[HIL: Snip]

unzvfu avatar Apr 17 '18 04:04 unzvfu

I've seen this during data upload as well:

Traceback (most recent call last):
  File "test_service.py", line 816, in <module>
    mapping_test(party1_filters[:size], party2_filters[:size], s1[:size], s2[:size])
  File "test_service.py", line 149, in mapping_test
    headers={'Authorization': new_map_response['update_tokens'][0]})
  File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 126, in put
    return request('put', url, data=data, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 490, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', timeout('The write operation timed out',))

I'm guessing I misunderstood how timeouts work in the requests library. I thought it was essentially how long to wait before giving up if nothing was happening. However it might be time allowed between hearing from the server which might explain my socket.timeout while uploading data.

hardbyte avatar Apr 17 '18 13:04 hardbyte