google-api-python-client
google-api-python-client copied to clipboard
googleapis
`socket.timeout: timed out` when using full IPv6 stack.
Environment details
- OS:
macOS 10.14.5 (18F132) Mojave - Python version:
3.7.3 - pip version:
19.1.1 -
google-api-python-clientversion:1.7.9
Steps to reproduce
Pre-steps:
- Have a full IPv6 stack with an IPv6 address. In my case, I have the following from Spectrum, through a Linksys WRT3200ACM router.
inet6 fe80::4ab:197d:b439:8d97%en0 prefixlen 64 secured scopeid 0x9
inet6 2604:6000:1013:a0eb:cca:87bb:d7ed:91b6 prefixlen 64 autoconf secured
inet6 2604:6000:1013:a0eb:91e1:1128:7728:8e1 prefixlen 64 autoconf temporary
Error:
- Follow the steps for the Python Quickstart (https://developers.google.com/docs/api/quickstart/python)
- run
python quickstart.py - python exits with
socket.timeout: timed outerror
If I open System Preferences > Network > Wi-FI > Advanced > TCP/IP and change the IPv6 configuration from Automatically to Link-local only, the quickstart.py properly runs.
A few folks have filed issues about socket timeouts recently - thank you for the additional info!
Socket problem is still here, in 2020.
Environment details
- OS: Alpine (docker image https://hub.docker.com/_/python
python:3.8.0-alpine)
Container runs on AWS EC2 instance.
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/rq/worker.py", line 886, in perform_job
rv = job.perform()
File "/usr/local/lib/python3.8/site-packages/rq/job.py", line 664, in perform
self._result = self._execute()
File "/usr/local/lib/python3.8/site-packages/rq/job.py", line 670, in _execute
return self.func(*self.args, **self.kwargs)
File "/usr/src/app/searchconsole/data.py", line 90, in load_data_for_property
save_data_batch(property, batch)
File "/usr/src/app/searchconsole/data.py", line 103, in save_data_batch
for data_batch in data_batches:
File "/usr/src/app/searchconsole/google/reports.py", line 103, in generate_lazy
response = request.execute()
File "/usr/local/lib/python3.8/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
return wrapped(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/googleapiclient/http.py", line 849, in execute
resp, content = _retry_request(
File "/usr/local/lib/python3.8/site-packages/googleapiclient/http.py", line 184, in _retry_request
raise exception
File "/usr/local/lib/python3.8/site-packages/googleapiclient/http.py", line 165, in _retry_request
resp, content = http.request(uri, method, *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/google_auth_httplib2.py", line 197, in request
response, content = self.http.request(
File "/usr/local/lib/python3.8/site-packages/httplib2/__init__.py", line 1982, in request
(response, content) = self._request(
File "/usr/local/lib/python3.8/site-packages/httplib2/__init__.py", line 1650, in _request
(response, content) = self._conn_request(
File "/usr/local/lib/python3.8/site-packages/httplib2/__init__.py", line 1589, in _conn_request
response = conn.getresponse()
File "/usr/local/lib/python3.8/http/client.py", line 1322, in getresponse
response.begin()
File "/usr/local/lib/python3.8/http/client.py", line 303, in begin
version, status, reason = self._read_status()
File "/usr/local/lib/python3.8/http/client.py", line 264, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/usr/local/lib/python3.8/socket.py", line 669, in readinto
return self._sock.recv_into(b)
File "/usr/local/lib/python3.8/ssl.py", line 1241, in recv_into
return self.read(nbytes, buffer)
File "/usr/local/lib/python3.8/ssl.py", line 1099, in read
return self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out
@vladignatyev - I had this error and was able to get around it by updating /etc/gai.conf to prefer IPV4 over IPV6 (uncomment the line precedence ::ffff:0:0/96 100 ).
I'm experiencing this right now, the whole weekend, a code that was previoudly fine now i can't get pass authentication:
.../python3.8/site-packages/httplib2/init.py, line 1324, in connect sock.connect((self.host.self.port__ socket.timetou: time out
Any advices how to solve this ?
I also started experiencing this issue just a day ago, with no modification to the code:
service = build('sheets', 'v4', credentials=get_creds())
File "/home/me/anaconda3/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper
return wrapped(*args, **kwargs)
File "/home/me/anaconda3/lib/python3.7/site-packages/googleapiclient/discovery.py", line 225, in build
requested_url, discovery_http, cache_discovery, cache, developerKey
File "/home/me/anaconda3/lib/python3.7/site-packages/googleapiclient/discovery.py", line 282, in _retrieve_discovery_doc
resp, content = http.request(actual_url)
File "/home/me/anaconda3/lib/python3.7/site-packages/httplib2/__init__.py", line 1991, in request
cachekey,
File "/home/me/anaconda3/lib/python3.7/site-packages/httplib2/__init__.py", line 1651, in _request
conn, request_uri, method, body, headers
File "/home/me/anaconda3/lib/python3.7/site-packages/httplib2/__init__.py", line 1557, in _conn_request
conn.connect()
File "/home/me/anaconda3/lib/python3.7/site-packages/httplib2/__init__.py", line 1324, in connect
sock.connect((self.host, self.port))
- OS: Ubuntu 18.04.5 LTS
- Python: 3.7.6
- pip: 20.0.2
- googleapiclient: 1.8.0
Uncommenting the line precedence ::ffff:0:0/96 100 in /etc/gai.conf fixed the problem.
My first attempt to reproduce on a Windows machine did not manifest the bug. Trying Linux....
Tried on Linux, haven't manifested the bug yet. Still working on it...
How often do you see the socket timeout? With every request? What fraction of your requests?
Has your code called socket.setdefaulttimeout(timeout) before calling this function?
What value is returned by socket.getdefaulttimeout() immediately before calling the API?
After studying the code in the client library, I don't see any red flags in the client library itself. An n8felton's stack trace demonstrates, the google client calls httplib2, which calls socket functions, where the timeout happens.
I'm trying to reproduce this issue on Alpine Linux running on an EC2 instance. When I run the quickstart (https://developers.google.com/docs/api/quickstart/python), it asks me to visit a localhost website to authenticate, which I can't do, since I'm connected via ssh with no graphical user interface.
Does someone have a simpler way to reproduce the misbehavior?
I'm having the same issue. Code was running fine a month ago, tried today and I get a timeout error.
[---------------------------------------------------------------------------
timeout Traceback (most recent call last)
<ipython-input-44-75c2f3716daf> in <module>()
----> 1 service = build('calendar','v3', credentials=credentials)
/Users/me/opt/anaconda3/lib/python3.7/site-packages/googleapiclient/_helpers.py in positional_wrapper(*args, **kwargs)
132 elif positional_parameters_enforcement == POSITIONAL_WARNING:
133 logger.warning(message)
--> 134 return wrapped(*args, **kwargs)
135
136 return positional_wrapper
/Users/me/opt/anaconda3/lib/python3.7/site-packages/googleapiclient/discovery.py in build(serviceName, version, http, discoveryServiceUrl, developerKey, model, requestBuilder, credentials, cache_discovery, cache, client_options)
223 options to set user options on the client.
224 (1) The API endpoint should be set through client_options. If API endpoint
--> 225 is not set, `GOOGLE_API_USE_MTLS_ENDPOINT` environment variable can be used
226 to control which endpoint to use.
227 (2) client_cert_source is not supported, client cert should be provided using
/Users/me/opt/anaconda3/lib/python3.7/site-packages/googleapiclient/discovery.py in _retrieve_discovery_doc(url, http, cache_discovery, cache, developerKey)
280 base=discovery_url,
281 http=http,
--> 282 developerKey=developerKey,
283 model=model,
284 requestBuilder=requestBuilder,
/Users/me/opt/anaconda3/lib/python3.7/site-packages/httplib2/__init__.py in request(self, uri, method, body, headers, redirections, connection_type)
1974 headers,
1975 redirections,
-> 1976 cachekey,
1977 )
1978 except Exception as e:
/Users/me/opt/anaconda3/lib/python3.7/site-packages/httplib2/__init__.py in _request(self, conn, host, absolute_uri, request_uri, method, body, headers, redirections, cachekey)
1638
1639 (response, content) = self._conn_request(
-> 1640 conn, request_uri, method, body, headers
1641 )
1642
/Users/me/opt/anaconda3/lib/python3.7/site-packages/httplib2/__init__.py in _conn_request(self, conn, request_uri, method, body, headers)
1544 try:
1545 if conn.sock is None:
-> 1546 conn.connect()
1547 conn.request(method, request_uri, body, headers)
1548 except socket.timeout:
/Users/me/opt/anaconda3/lib/python3.7/site-packages/httplib2/__init__.py in connect(self)
1315 if has_timeout(self.timeout):
1316 sock.settimeout(self.timeout)
-> 1317 sock.connect((self.host, self.port))
1318
1319 self.sock = self._context.wrap_socket(sock, server_hostname=self.host)
timeout: timed out](url)
If another case is helpful, I'm getting the same error when attempting to instantiate a storagetransfer client, running from GKE. From my laptop, using the same docker image, I can't repro.
File "/usr/local/lib/python3.8/site-packages/{...}/load.py", line 137, in _storage_transfer_client
return googleapiclient.discovery.build("storagetransfer", "v1", cache=MemoryCache())
File "/usr/local/lib/python3.8/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper
return wrapped(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/googleapiclient/discovery.py", line 245, in build
content = _retrieve_discovery_doc(
File "/usr/local/lib/python3.8/site-packages/googleapiclient/discovery.py", line 305, in _retrieve_discovery_doc
resp, content = http.request(actual_url)
File "/usr/local/lib/python3.8/site-packages/httplib2/__init__.py", line 1985, in request
(response, content) = self._request(
File "/usr/local/lib/python3.8/site-packages/httplib2/__init__.py", line 1650, in _request
(response, content) = self._conn_request(
File "/usr/local/lib/python3.8/site-packages/httplib2/__init__.py", line 1589, in _conn_request
response = conn.getresponse()
File "/usr/local/lib/python3.8/http/client.py", line 1347, in getresponse
response.begin()
File "/usr/local/lib/python3.8/http/client.py", line 307, in begin
version, status, reason = self._read_status()
File "/usr/local/lib/python3.8/http/client.py", line 268, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/usr/local/lib/python3.8/socket.py", line 669, in readinto
return self._sock.recv_into(b)
File "/usr/local/lib/python3.8/ssl.py", line 1241, in recv_into
return self.read(nbytes, buffer)
File "/usr/local/lib/python3.8/ssl.py", line 1099, in read
return self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out
My problem was solved by duplicating my GKE cluster, and it working on the new GKE cluster. So not sure what the actual problem was — calling build with any valid service python -c 'import googleapiclient.discovery; googleapiclient.discovery.build("compute", "v1")' would hang before timing out.
Thank you @max-sixty, that's a really useful data point. I will follow up with GKE engineers. Is the old, malfunctioning cluster still running?
Hi @SurferJeffAtGoogle — this ended up being a GKE issue, and the "it works on another cluster" was because the problem hadn't spread to another zone at that point; nothing to do with the cluster itself (it later spread to all zones).
I can't see it in the Incident History, but it was listed under https://console.cloud.google.com/support/cases;tab=knownissues for a while. Our Google support case number was 25392673. Given it spanned half a dozen regions, I would have expected to see it in the Incident History.
Let me know any questions! This was fairly painful for us — seemingly more so than for others, I still haven't fully resolved why.
@max-sixty , can you tell us if you are still seeing the timeout issue? There was an issue related to API discovery service between 2020-10-06 to 2020-10-08 from GCE VMs which may have caused this timeout issue as well and the changes were rolled back to mitigate the issue. So want to check if it is still happening. Please advice.
This was fixed with the rollback and is no longer an issue for us, I should have made that clearer above.
There was an issue related to API discovery service between 2020-10-06 to 2020-10-08 from GCE VMs which may have caused this timeout issue as well and the changes were rolled back to mitigate the issue.
I can't see this on the incident history, I'm guessing you see this internally
There have been recent changes to our network configuration. Would someone who has experienced this issue please confirm it's still happening?
I just started getting this issue 2 days ago. I'm using the Google Custom Search API. Does anyone know how I can solve this?
Traceback (most recent call last): File "serp_googlecs.py", line 73, in <module> format_search_results('75_dataset.csv') File "serp_googlecs.py", line 56, in format_search_results queries, names = combined_serp_cs(data) File "serp_googlecs.py", line 50, in combined_serp_cs res = google_api_search(query, my_api_key, my_cse_id) File "serp_googlecs.py", line 32, in google_api_search res = service.cse().list(q=search_term, cx=cse_id, **kwargs).execute() File "/Users/h35528/opt/anaconda3/envs/py37/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper return wrapped(*args, **kwargs) File "/Users/h35528/opt/anaconda3/envs/py37/lib/python3.7/site-packages/googleapiclient/http.py", line 901, in execute headers=self.headers, File "/Users/h35528/opt/anaconda3/envs/py37/lib/python3.7/site-packages/googleapiclient/http.py", line 204, in _retry_request raise exception File "/Users/h35528/opt/anaconda3/envs/py37/lib/python3.7/site-packages/googleapiclient/http.py", line 177, in _retry_request resp, content = http.request(uri, method, *args, **kwargs) File "/Users/h35528/opt/anaconda3/envs/py37/lib/python3.7/site-packages/httplib2/__init__.py", line 1994, in request cachekey, File "/Users/h35528/opt/anaconda3/envs/py37/lib/python3.7/site-packages/httplib2/__init__.py", line 1651, in _request conn, request_uri, method, body, headers File "/Users/h35528/opt/anaconda3/envs/py37/lib/python3.7/site-packages/httplib2/__init__.py", line 1557, in _conn_request conn.connect() File "/Users/h35528/opt/anaconda3/envs/py37/lib/python3.7/site-packages/httplib2/__init__.py", line 1324, in connect sock.connect((self.host, self.port)) socket.timeout: timed out
I just got it, previously working code now does not pass the auth.
File "lib/python3.8/site-packages/httplib2/__init__.py", line 1324, in connect sock.connect((self.host, self.port)) socket.timeout: timed out
Any fix for this?
Edit: Uncommenting the line worked for me on Ubuntu 20
I also started experiencing this issue just a day ago, with no modification to the code:
service = build('sheets', 'v4', credentials=get_creds()) File "/home/me/anaconda3/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper return wrapped(*args, **kwargs) File "/home/me/anaconda3/lib/python3.7/site-packages/googleapiclient/discovery.py", line 225, in build requested_url, discovery_http, cache_discovery, cache, developerKey File "/home/me/anaconda3/lib/python3.7/site-packages/googleapiclient/discovery.py", line 282, in _retrieve_discovery_doc resp, content = http.request(actual_url) File "/home/me/anaconda3/lib/python3.7/site-packages/httplib2/__init__.py", line 1991, in request cachekey, File "/home/me/anaconda3/lib/python3.7/site-packages/httplib2/__init__.py", line 1651, in _request conn, request_uri, method, body, headers File "/home/me/anaconda3/lib/python3.7/site-packages/httplib2/__init__.py", line 1557, in _conn_request conn.connect() File "/home/me/anaconda3/lib/python3.7/site-packages/httplib2/__init__.py", line 1324, in connect sock.connect((self.host, self.port))
- OS: Ubuntu 18.04.5 LTS
- Python: 3.7.6
- pip: 20.0.2
- googleapiclient: 1.8.0
Uncommenting the line
precedence ::ffff:0:0/96 100in /etc/gai.conf fixed the problem.
@parthea If you're seeking a very challenging bug to investigate, try this one!
@nulld mentioned a workaround in #563 where he had to move the socket.setdefaulttimeout() function before import googleapiclient. If it solves the problem, then credit goes to @nulld ! Either way, we can use this data point to better understand whether there are multiple issues going on. Please can someone who is experiencing the issue try the workaround and report back?
I think I see the same issue, but it's not related to ipv6, or the python library.
I found this curl to reproduce it:
curl --http1.1 -v --request GET 'https://analytics.googleapis.com/analytics/v3/metadata/ga/columns?alt=json' --header 'Authorization: Bearer ya29.... -v
It usually works without problems, but about every 3rd or 4th request gets "stuck" and never finishes.
I'm running this command inside of a container based on python:3.6-slim, inside of a kubernetes cluster with nodes in europe-west-3. It's the same with http1.1 and http2
The logs of a successfull request look like this:
curl -4 --http2 --request GET 'https://analytics.googleapis.com/analytics/v3/metadata/ga/columns?alt=json' --header 'Authorization: Bearer ya29....' -v > /dev/null
Note: Unnecessary use of -X or --request, GET is already inferred.
* Expire in 0 ms for 6 (transfer 0x55560bd5af90)
* Expire in 1 ms for 1 (transfer 0x55560bd5af90)
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Expire in 0 ms for 1 (transfer 0x55560bd5af90)
* Expire in 2 ms for 1 (transfer 0x55560bd5af90)
* Expire in 0 ms for 1 (transfer 0x55560bd5af90)
* Expire in 0 ms for 1 (transfer 0x55560bd5af90)
* Expire in 2 ms for 1 (transfer 0x55560bd5af90)
* Expire in 0 ms for 1 (transfer 0x55560bd5af90)
* Expire in 1 ms for 1 (transfer 0x55560bd5af90)
* Expire in 2 ms for 1 (transfer 0x55560bd5af90)
* Expire in 1 ms for 1 (transfer 0x55560bd5af90)
* Expire in 1 ms for 1 (transfer 0x55560bd5af90)
* Expire in 2 ms for 1 (transfer 0x55560bd5af90)
* Expire in 1 ms for 1 (transfer 0x55560bd5af90)
* Expire in 1 ms for 1 (transfer 0x55560bd5af90)
* Expire in 2 ms for 1 (transfer 0x55560bd5af90)
* Expire in 1 ms for 1 (transfer 0x55560bd5af90)
* Expire in 1 ms for 1 (transfer 0x55560bd5af90)
* Expire in 2 ms for 1 (transfer 0x55560bd5af90)
* Expire in 2 ms for 1 (transfer 0x55560bd5af90)
* Expire in 2 ms for 1 (transfer 0x55560bd5af90)
* Expire in 4 ms for 1 (transfer 0x55560bd5af90)
* Expire in 2 ms for 1 (transfer 0x55560bd5af90)
* Expire in 2 ms for 1 (transfer 0x55560bd5af90)
* Expire in 4 ms for 1 (transfer 0x55560bd5af90)
* Expire in 3 ms for 1 (transfer 0x55560bd5af90)
* Expire in 3 ms for 1 (transfer 0x55560bd5af90)
* Expire in 4 ms for 1 (transfer 0x55560bd5af90)
* Expire in 4 ms for 1 (transfer 0x55560bd5af90)
* Expire in 4 ms for 1 (transfer 0x55560bd5af90)
* Expire in 5 ms for 1 (transfer 0x55560bd5af90)
* Trying 172.217.16.138...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x55560bd5af90)
* Connected to analytics.googleapis.com (172.217.16.138) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ [15 bytes data]
* TLSv1.3 (IN), TLS handshake, Certificate (11):
{ [2633 bytes data]
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
{ [78 bytes data]
* TLSv1.3 (IN), TLS handshake, Finished (20):
{ [52 bytes data]
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.3 (OUT), TLS handshake, Finished (20):
} [52 bytes data]
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
* subject: C=US; ST=California; L=Mountain View; O=Google LLC; CN=upload.video.google.com
* start date: Nov 10 14:39:49 2020 GMT
* expire date: Feb 2 14:39:48 2021 GMT
* subjectAltName: host "analytics.googleapis.com" matched cert's "*.googleapis.com"
* issuer: C=US; O=Google Trust Services; CN=GTS CA 1O1
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
} [5 bytes data]
* Using Stream ID: 1 (easy handle 0x55560bd5af90)
} [5 bytes data]
> GET /analytics/v3/metadata/ga/columns?alt=json HTTP/2
> Host: analytics.googleapis.com
> User-Agent: curl/7.64.0
> Accept: */*
> Authorization: Bearer ya29.a...
>
{ [5 bytes data]
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
} [5 bytes data]
< HTTP/2 200
< content-type: application/json; charset=UTF-8
< vary: X-Origin
< vary: Referer
< vary: Origin,Accept-Encoding
< date: Tue, 05 Jan 2021 14:11:13 GMT
< server: ESF
< cache-control: private
< x-xss-protection: 0
< x-frame-options: SAMEORIGIN
< x-content-type-options: nosniff
< accept-ranges: none
<
and a request that was "stuck":
curl -4 --http2 --request GET 'https://analytics.googleapis.com/analytics/v3/metadata/ga/columns?alt=json' --header 'Authorization: Bearer ya29....' -v > /dev/null
Note: Unnecessary use of -X or --request, GET is already inferred.
* Expire in 0 ms for 6 (transfer 0x56189cda3f90)
* Expire in 1 ms for 1 (transfer 0x56189cda3f90)
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Expire in 0 ms for 1 (transfer 0x56189cda3f90)
* Expire in 2 ms for 1 (transfer 0x56189cda3f90)
* Expire in 0 ms for 1 (transfer 0x56189cda3f90)
* Expire in 0 ms for 1 (transfer 0x56189cda3f90)
* Expire in 2 ms for 1 (transfer 0x56189cda3f90)
* Expire in 0 ms for 1 (transfer 0x56189cda3f90)
* Expire in 0 ms for 1 (transfer 0x56189cda3f90)
* Expire in 2 ms for 1 (transfer 0x56189cda3f90)
* Expire in 1 ms for 1 (transfer 0x56189cda3f90)
* Expire in 1 ms for 1 (transfer 0x56189cda3f90)
* Expire in 2 ms for 1 (transfer 0x56189cda3f90)
* Expire in 1 ms for 1 (transfer 0x56189cda3f90)
* Expire in 1 ms for 1 (transfer 0x56189cda3f90)
* Expire in 2 ms for 1 (transfer 0x56189cda3f90)
* Expire in 1 ms for 1 (transfer 0x56189cda3f90)
* Expire in 1 ms for 1 (transfer 0x56189cda3f90)
* Expire in 2 ms for 1 (transfer 0x56189cda3f90)
* Expire in 2 ms for 1 (transfer 0x56189cda3f90)
* Expire in 2 ms for 1 (transfer 0x56189cda3f90)
* Expire in 2 ms for 1 (transfer 0x56189cda3f90)
* Trying 172.217.22.42...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x56189cda3f90)
* Connected to analytics.googleapis.com (172.217.22.42) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ [15 bytes data]
* TLSv1.3 (IN), TLS handshake, Certificate (11):
{ [2633 bytes data]
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
{ [80 bytes data]
* TLSv1.3 (IN), TLS handshake, Finished (20):
{ [52 bytes data]
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.3 (OUT), TLS handshake, Finished (20):
} [52 bytes data]
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
* subject: C=US; ST=California; L=Mountain View; O=Google LLC; CN=upload.video.google.com
* start date: Nov 10 14:39:49 2020 GMT
* expire date: Feb 2 14:39:48 2021 GMT
* subjectAltName: host "analytics.googleapis.com" matched cert's "*.googleapis.com"
* issuer: C=US; O=Google Trust Services; CN=GTS CA 1O1
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
} [5 bytes data]
* Using Stream ID: 1 (easy handle 0x56189cda3f90)
} [5 bytes data]
> GET /analytics/v3/metadata/ga/columns?alt=json HTTP/2
> Host: analytics.googleapis.com
> User-Agent: curl/7.64.0
> Accept: */*
> Authorization: Bearer ya29....
>
{ [5 bytes data]
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
} [5 bytes data]
Had the same issue. I can see from a nmap that TCP ports 80 and 443 are not open on the IPv6 addresses for www.googleapis.com but they are open for IPv4 addresses:
$ nmap -Pn -p80,443 -6 www.googleapis.com
Starting Nmap 7.60 ( https://nmap.org ) at 2021-06-04 17:57 AEST
Nmap scan report for www.googleapis.com (2404:6800:4015:802::200a)
Host is up.
Other addresses for www.googleapis.com (not scanned): 2404:6800:4015:801::200a 2404:6800:4015:800::200a 2404:6800:4015:803::200a 142.250.70.234 142.250.70.170 142.250.70.138 142.250.70.202
rDNS record for 2404:6800:4015:802::200a: mel05s01-in-x0a.1e100.net
PORT STATE SERVICE
80/tcp filtered http
443/tcp filtered https
Nmap done: 1 IP address (1 host up) scanned in 3.07 seconds
$ nmap -4 www.googleapis.com
Starting Nmap 7.60 ( https://nmap.org ) at 2021-06-04 17:56 AEST
Nmap scan report for www.googleapis.com (142.250.70.202)
Host is up (0.034s latency).
Other addresses for www.googleapis.com (not scanned): 2404:6800:4015:802::200a 2404:6800:4015:803::200a 2404:6800:4015:800::200a 2404:6800:4015:801::200a 142.250.70.234 142.250.70.138 142.250.70.170
rDNS record for 142.250.70.202: mel05s01-in-f10.1e100.net
Not shown: 998 filtered ports
PORT STATE SERVICE
80/tcp open http
443/tcp open https
Nmap done: 1 IP address (1 host up) scanned in 5.16 seconds
I used the quick hack from here in my code to force the underlying socket to connect over IPv4: https://stackoverflow.com/a/50044152
It looks like a Google firewall issue
Hi there! Just came to say that this issue (IPv6 connections that remain stuck until they timeout) still happens from time to time.
Last time I experienced it was on Thursday 7 November morning from ~9:00 to 10:15 (UTC+1) trying to connect to the address resolved by accounts.google.com.
