Cannot Fetch from Google Scholar

[swhussain110]

I am using scholarly==1.7.11

and shows the following exception since a week.

MaxTriesExceededException("Cannot Fetch from Google Scholar.") scholarly._proxy_generator.MaxTriesExceededException: Cannot Fetch from Google Scholar.

[abc12399]

i have the same problem with you

[JoseHervas]

Same issue here, with scholarly==1.7.11

The method search_pubs works perfectly. The method search_pubs_custom_url returns the MaxTriesExceededException even with Proxy.

[bajinsheng]

I have encountered the same issue. Any solution?

[arunkannawadi]

@JoseHervas can you post a minimal working example?

[jason-huanghao]

I have the same Issure of reaching, I used the sleep method which allows me running the code without problem for at least 5 hours.

import time 

pg.FreeProxies()  # refresh the proxy list
scholarly.use_proxy(pg)  # use the new proxy list
 try:
          search_query = scholarly.search_author(name)
         time.sleep(0.08)
 except:
         print("Max tries")

[hyperbolic-c]

I got the same error. The logging info is

2024-03-18 21:19:21,365 - scholarly - INFO - Enabling proxies: http=http://127.0.0.1:10809/ https=http://127.0.0.1:10809/
2024-03-18 21:19:22,633 - scholarly - INFO - Proxy works! IP address: 23.26.254.192
2024-03-18 21:19:23,071 - scholarly - INFO - Proxy setup successfully
Proxy setup sucess: True.
2024-03-18 21:19:25,159 - scholarly - INFO - Proxy works! IP address: 23.26.254.192
2024-03-18 21:19:25,858 - scholarly - INFO - Proxy works! IP address: 23.26.254.192
2024-03-18 21:19:28,435 - scholarly - INFO - Getting https://scholar.google.com/scholar?hl=en&q=A%20lightweight%20network%20for%20photovoltaic%20cell%20defect%20detection%20in%20electroluminescence%20images%20based%20on%20neural%20architecture%20search%20and%20knowledge%20distillation&as_vis=0&as_sdt=0,33
2024-03-18 21:19:32,429 - scholarly - INFO - Exception ConnectError while fetching page: ('[Errno 11001] getaddrinfo failed',)
2024-03-18 21:19:32,430 - scholarly - INFO - Retrying with a new session.
2024-03-18 21:19:36,985 - scholarly - INFO - Exception ConnectError while fetching page: ('[Errno 11001] getaddrinfo failed',)
2024-03-18 21:19:36,985 - scholarly - INFO - Retrying with a new session.

and then

[288](file:///D:/miniconda/envs/research/lib/site-packages/scholarly/_navigator.py:288) def search_publications(self, url: str) -> _SearchScholarIterator:
    [289](file:///D:/miniconda/envs/research/lib/site-packages/scholarly/_navigator.py:289)     """Returns a Publication Generator given a url
    [290](file:///D:/miniconda/envs/research/lib/site-packages/scholarly/_navigator.py:290) 
    [291](file:///D:/miniconda/envs/research/lib/site-packages/scholarly/_navigator.py:291)     :param url: the url where publications can be found.
...
    [188](file:///D:/miniconda/envs/research/lib/site-packages/scholarly/_navigator.py:188)     return self._get_page(pagerequest, True)
    [189](file:///D:/miniconda/envs/research/lib/site-packages/scholarly/_navigator.py:189) else:
--> [190](file:///D:/miniconda/envs/research/lib/site-packages/scholarly/_navigator.py:190)     raise MaxTriesExceededException("Cannot Fetch from Google Scholar.")

I would appreciate it if angbody have any solutions. Thanks