fanfictionnet_ff_proxy
fanfictionnet_ff_proxy copied to clipboard
Never ending loop in the proxy
Not sure whether this is a duplicate of #12 , but when pull in 8aeabc4 the situation became substantially worse. Whereas before (for example with 9879f07ef293010f7f2ff8e232c94393880236a6) fanficfare just once crashed and next time it worked fine, now the proxy gets into some kind of neverending loop and the result is crash of fanficfare:
fun~/K/f/t/austen$ fanficfare https://www.fanfiction.net/s/14191300
Traceback (most recent call last):
File "/home/matej/.bin/fanficfare", line 8, in <module>
sys.exit(main())
^^^^^^
File "/home/matej/.local/lib/python3.11/site-packages/fanficfare/cli.py", line 344, in main
dispatch(options, urls, passed_defaultsini, passed_personalini, warn, fail)
File "/home/matej/.local/lib/python3.11/site-packages/fanficfare/cli.py", line 320, in dispatch
do_download(url,
File "/home/matej/.local/lib/python3.11/site-packages/fanficfare/cli.py", line 435, in do_download
adapter.getStoryMetadataOnly()
File "/home/matej/.local/lib/python3.11/site-packages/fanficfare/adapters/base_adapter.py", line 327, in getStoryMetadataOnly
self.doExtractChapterUrlsAndMetadata(get_cover=get_cover)
File "/home/matej/.local/lib/python3.11/site-packages/fanficfare/adapters/adapter_fanfictionnet.py", line 113, in doExtractChapterUrlsAndMetadata
data = self.get_request(url)
^^^^^^^^^^^^^^^^^^^^^
File "/home/matej/.local/lib/python3.11/site-packages/fanficfare/requestable.py", line 119, in get_request
return self.get_request_redirected(url,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/matej/.local/lib/python3.11/site-packages/fanficfare/requestable.py", line 111, in get_request_redirected
(data,rurl) = self.configuration.get_fetcher().get_request_redirected(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/matej/.local/lib/python3.11/site-packages/fanficfare/fetchers/base_fetcher.py", line 133, in get_request_redirected
fetchresp = self.do_request('GET',
^^^^^^^^^^^^^^^^^^^^^^
File "/home/matej/.local/lib/python3.11/site-packages/fanficfare/fetchers/decorators.py", line 68, in fetcher_do_request
fetchresp = chainfn(
^^^^^^^^
File "/home/matej/.local/lib/python3.11/site-packages/fanficfare/fetchers/cache_basic.py", line 122, in fetcher_do_request
fetchresp = chainfn(
^^^^^^^^
File "/home/matej/.local/lib/python3.11/site-packages/fanficfare/fetchers/decorators.py", line 102, in fetcher_do_request
fetchresp = chainfn(
^^^^^^^^
File "/home/matej/.local/lib/python3.11/site-packages/fanficfare/fetchers/base_fetcher.py", line 106, in do_request
fetchresp = self.request(method,url,
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/matej/.local/lib/python3.11/site-packages/fanficfare/fetchers/fetcher_nsapa_proxy.py", line 202, in request
raise exceptions.FailedToDownload(
fanficfare.exceptions.FailedToDownload: nsapa_proxy: reply still truncated after 5 retry
fun~/K/f/t/austen$
When running the proxy as python3 chrome_content.py --verbose --write-log --log-filename /tmp/fanfiction_proxy_log.txt
this is fanfiction_proxy_log.txt.