ERROR:root:Error retrieving https://twitter.com/: ConnectionError(MaxRetryError("HTTPSConnectionPool(host='twitter.com', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fae534b7a60>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))"))

[daviitp]

Initial Check

If the issue is a request please specify that it is a request in the title (Example: [REQUEST] more features). If this is a question regarding 'twint' please specify that it's a question in the title (Example: [QUESTION] What is x?). Please only submit issues related to 'twint'. Thanks.

Make sure you've checked the following:

• [] Python version is 3.6; ---> 3.8.5
• [] Updated Twint with pip3 install --user --upgrade -e git+https://github.com/twintproject/twint.git@origin/master#egg=twint; Yes (twint 2.1.21)
• [] I have searched the issues and there are no duplicates of this issue/question/request. ---> Yes

Command Ran

Please provide the exact command ran including the username/search/code so I may reproduce the issue.

./test.py

Code:

#!/usr/bin/env python3

import twint

c = twint.Config() c.Since = "2021-01-01" c.Proxy_type = 'http' c.Proxy_host = '10.30.x.x' c.Proxy_port = 80 c.Search = 'milanesa' # (it's just a test, but i'm a big fan of "milanesas") twint.run.Search(c)

Description of Issue

Please use as much detail as possible.

Hi! I'm trying to run twint behind a corporate proxy, but it's not working. When running in my laptop (without a proxy) it works fine. The error is easy: twint is not able to resolve twitter.com, but the proxy is who has to resolve.

Running a tcpdump (with the proxy host as filter) and test.py at the same time shows nothing in the traffic capture, so it seems to not going out through the proxy.

The debug shows the following:

DEBUG:2021-01-29 18:03:12,000:root:twint.run:Search DEBUG:2021-01-29 18:03:12,001:root:twint.run:run DEBUG:2021-01-29 18:03:12,001:asyncio:Using selector: EpollSelector DEBUG:2021-01-29 18:03:12,002:root:twint.run:Twint:init DEBUG:2021-01-29 18:03:12,002:root:Retrieving guest token DEBUG:2021-01-29 18:03:12,003:root:Retrieving https://twitter.com/ DEBUG:2021-01-29 18:03:12,003:urllib3.connectionpool:Starting new HTTPS connection (1): twitter.com:443 WARNING:2021-01-29 18:03:12,016:root:Error retrieving https://twitter.com/: ConnectionError(MaxRetryError("HTTPSConnectionPool(host='twitter.com', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fe585a1fac0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))")), retrying ---> 6 times CRITICAL:2021-01-29 18:04:14,153:root:6 requests to https://twitter.com failed, giving up.

I have read a few (unofficial) comments in other issues/questions saying proxy is not working after a former API update, so maybe that is the simple answer.

Am I doing something wrong? Or twint is not actually working behind a proxy server?

Thanks in advance!

Environment Details

Using Windows, Linux? What OS version? Running this in Anaconda? Jupyter Notebook? Terminal?

Ubuntu 20.04.1 LTS, Python 3.8.5

[CharleoY]

How many instances did you run concurrently? Maybe because the number of connection is too high to trigger the protection protocol of twitter's server?

[CharleoY]

I had the same issue under Ubuntu 20.04, but it works fine in Windows10, one sign of this issue occurring is that all of in and out traffic are disappeared, and you can't open any webpage.

[Skyorca]

I face the same error using proxy after updating twint with git...

[jecs89]

This problem can be related to the number of tweets you collected, there is a limit for IP address.

[dpakpdl]

The proxy is not currently working. I have a fix for it. Check it

[theCreativitist]

I had the same problem. fix? changing the proxy. apparently the proxy was blocked by twitter servers.