lightnovel-crawler
lightnovel-crawler copied to clipboard
Fix this source : Unknown
Let us know
Novel URL: the girl who ate a death god App Location: All App Version: 2.35
Describe this issue
Some source search is returning a novelhard.com
url, which there is no crawler for.
Something redirects to novelhard. I am aware of this problem, but I do not know how to find the culprit.
Okay, I found a way to save the log to a file. @dipu-bd You need to add a setting to save logs to a file. I can't fit everything in the cmd buffer.
2022-09-23 10:51:04,528 [DEBUG] (lncrawl.core.novel_search)
[{'title': 'Into The Kaleidoscope', 'url': 'https://novelhard.com/novel/into-the-kaleidoscope/', 'info': 'Chapter 16 | Rating: 0'}, {'title': 'The Villainess That Was Once A VRMMO', 'url': 'https://novelhard.com/novel/the-villainess-that-was-once-a-vrmmo/', 'info': 'Chapter 49.2 | Rating: 0'}, {'title': 'I Have 7 Big Brothers', 'url': 'https://novelhard.com/novel/i-have-7-big-brothers/', 'info': 'Chapter 26 | Rating: 0'}, {'title': 'Leveling Up My Husband to the Max', 'url': 'https://novelhard.com/novel/leveling-up-my-husband-to-the-max/', 'info': 'Chapter 54 | Rating: 0'}, {'title': 'The cursed prince’s strange bride', 'url': 'https://novelhard.com/novel/the-cursed-princes-strange-bride/', 'info': 'Chapter 29 | Rating: 0'}, {'title': 'After the Full-Level Boss Entered the Infinite Game By Mistake', 'url': 'https://novelhard.com/novel/after-the-full-level-boss-entered-the-infinite-game-by-mistake/', 'info': 'Chapter 72.1 | Rating: 0'}, {'title': 'The Regressor and the Blind Saint', 'url': 'https://novelhard.com/novel/the-regressor-and-the-blind-saint/', 'info': 'Chapter 29 | Rating: 0'}, {'title': 'The Villains’ Pampering Is Too Great!', 'url': 'https://novelhard.com/novel/the-villains-pampering-is-too-great/', 'info': 'Chapter 57 | Rating: 0'}, {'title': 'Farming in the Mountains: Max Level Jiaojiao Is Three Years Old', 'url': 'https://novelhard.com/novel/farming-in-the-mountains-max-level-jiaojiao-is-three-years-old/', 'info': 'Chapter 76 | Rating: 0'}, {'title': 'I’m the Villain’s Constellation but I’m Broke', 'url': 'https://novelhard.com/novel/im-the-villains-constellation-but-im-broke/', 'info': 'Chapter 10 | Rating: 0'}]
2022-09-23 10:51:04,531 [DEBUG] (lncrawl.core.crawler)
HTTPSConnectionPool(host='overabook.com', port=443): Max retries exceeded with url: /?s=the&post_type=wp-manga&author=&artist=&release= (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x0000000017134190>: Failed to establish a new connection: [Errno 11004] getaddrinfo failed')) | Retrying...
2022-09-23 10:51:04,536 [INFO] (lncrawl.core.novel_search)
10 results from https://noveltrench.com/
https://noveltrench.com/ -> https://novelhard.com/