ecmwf-api-client
ecmwf-api-client copied to clipboard
ecmwf
Intermittent "Connection reset by peer" when downloading from MARS via api
I get "Connection reset by peer" randomly when downloading ECMWF files from MARS via the python api. I have a script that submits 20 at a time (max MARS queue limit). Most of the time, it works as expected and chunks through files. But sometimes (once a week) it throws the below connection error.
File ".........pynio3.6/lib/python3.6/ssl.py", line 631, in read
v = self._sslobj.read(len, buffer)
ConnectionResetError: [Errno 104] Connection reset by peer
Does anyone have any suggestions for how to troubleshoot this error?
We've also been experiencing this issue quite frequently in recent months.
HTTP Error 500: Internal Server Error
WARNING: HTTP received 500
{"code": 500, "error": "Error opening socket: a socket error occurred"}
HTTP Error 503: Service Unavailable
WARNING: HTTP received 503
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>503 Service Unavailable</title>
</head><body>
<h1>Service Unavailable</h1>
<p>The server is temporarily unable to service your
request due to maintenance downtime or capacity
problems. Please try again later.</p>
</body></html>
HTTP Error 500: Internal Server Error
WARNING: HTTP received 500
{"code": 500, "error": "Error opening socket: a socket error occurred"}
Unexpected error: <class 'socket.error'>
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/ecmwfapi/api.py", line 133, in wrapped
return func(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/ecmwfapi/api.py", line 250, in call
res = opener.open(req)
File "/usr/lib64/python2.7/urllib2.py", line 429, in open
response = self._open(req, data)
File "/usr/lib64/python2.7/urllib2.py", line 447, in _open
'_open', req)
File "/usr/lib64/python2.7/urllib2.py", line 407, in _call_chain
result = func(*args)
File "/usr/lib64/python2.7/urllib2.py", line 1241, in https_open
context=self._context)
File "/usr/lib64/python2.7/urllib2.py", line 1201, in do_open
r = h.getresponse(buffering=True)
File "/usr/lib64/python2.7/httplib.py", line 1121, in getresponse
response.begin()
File "/usr/lib64/python2.7/httplib.py", line 438, in begin
version, status, reason = self._read_status()
File "/usr/lib64/python2.7/httplib.py", line 394, in _read_status
line = self.fp.readline(_MAXLINE + 1)
File "/usr/lib64/python2.7/socket.py", line 480, in readline
data = self._sock.recv(self._rbufsize)
File "/usr/lib64/python2.7/ssl.py", line 772, in recv
return self.read(buflen)
File "/usr/lib64/python2.7/ssl.py", line 659, in read
v = self._sslobj.read(len)
error: [Errno 104] Connection reset by peer
Same here. I've been getting this error frequently during the previous few days. Can't finish the retrieval task and I have to start over again as the 3rd party tool I'm using deletes old data every time it starts. This basics forces me to break my retrieval into small pieces.
