Handling of Failed URL Downloads

[x9906789x]

Describe the bug I have spent a couple days trying to determine this issue and I'm sure it is simple but I'm at a loss. When NZBGet has an error or fails a download when it is a URL download it does not report it back to Medusa as a failure which in turn leaves Medusa with a file that is snatched in perpetuity. This appears to only be happening when a URL is passed to NZBGet and not the nzb file. The status returns as Failure when its a URL as opposed to bad when it is a nzb file that is passed to NZBGet. I would be fine if I could find a setting in either Medusa or NZBGet to not use URL but I haven't been able to locate that either. I know NZBGet has URL settings but none seem to stop Medusa from sending it URLs. They just sit in queue. I expected a post processing script to run after the failure so that Medusa knows it failed but i can't find that that occurred in the logs. This is running on Linux. Python 2.7.

I just tried to pull logs of all three of these instances and they do not appear in the logs. They only appear in the webgui of NZBGet which could just be my oversight at this point.

below is the last log which is before the most recent time stamp on that image i attached. 2019-05-31 19:50:56 INFO ::MAIN: The /root/nzbget/scripts/nzbToSickBeard.py script completed successfully.

and below is the log that spans the second time stamp of that url failure where nothing appears. 2019-05-31 19:48:30 INFO ::MAIN: Checking if git needs an update 2019-05-31 19:48:31 INFO ::MAIN: No update needed 2019-05-31 19:48:31 INFO ::MAIN: nzbToMedia Version:0db7c3e10c01cee85814a31fd3c5232b1d7d144e Branch:master (Linux 4.15.0-50-generic) 2019-05-31 19:48:31 INFO ::MAIN: ######################################################### 2019-05-31 19:48:31 INFO ::MAIN: ## ..::[nzbToMedia.py]::.. ## 2019-05-31 19:48:31 INFO ::MAIN: ######################################################### 2019-05-31 19:48:31 INFO ::MAIN: Script triggered from NZBGet Version 21.0. 2019-05-31 19:48:31 INFO ::MAIN: Download failed with status FAILURE/HEALTH. 2019-05-31 19:48:31 INFO ::MAIN: Auto-detected SECTION:SickBeard 2019-05-31 19:48:31 INFO ::MAIN: Calling SickBeard:TV to post-process:Last.Week.Tonight.with.John.Oliver.S06E12.720p.AMZN.WEB-DL.DDP2.0.H.264-monkee 2019-05-31 19:48:31 INFO ::MAIN: Attempting to auto-detect TV fork 2019-05-31 19:48:31 INFO ::MAIN: SickBeard:TV fork auto-detection successful ... 2019-05-31 19:48:31 INFO ::MAIN: SickBeard:TV fork set to Medusa-api 2019-05-31 19:48:31 WARNING ::SICKBEARD: No media files found in directory /root/nzbget/downloads/intermediate/Last.Week.Tonight.with.John.Oliver.S06E12.720p.AMZN.WEB-DL.DDP2.0.H.264-monkee.#2357. Processing this as a failed download 2019-05-31 19:48:31 POSTPROCESS::SICKBEARD: FAILED: The download failed. Sending 'failed' process request to Medusa-api branch 2019-05-31 19:48:31 POSTPROCESS::SICKBEARD: Deleting failed files and folder /root/nzbget/downloads/intermediate/Last.Week.Tonight.with.John.Oliver.S06E12.720p.AMZN.WEB-DL.DDP2.0.H.264-monkee.#2357 2019-05-31 19:48:31 INFO ::MAIN: Deleting /root/nzbget/downloads/intermediate/Last.Week.Tonight.with.John.Oliver.S06E12.720p.AMZN.WEB-DL.DDP2.0.H.264-monkee.#2357 2019-05-31 19:48:31 ERROR ::MAIN: Unable to delete folder /root/nzbget/downloads/intermediate/Last.Week.Tonight.with.John.Oliver.S06E12.720p.AMZN.WEB-DL.DDP2.0.H.264-monkee.#2357 2019-05-31 19:48:31 INFO ::CLEANDIR: Directory /root/nzbget/downloads/intermediate/Last.Week.Tonight.with.John.Oliver.S06E12.720p.AMZN.WEB-DL.DDP2.0.H.264-monkee.#2357 has been processed and removed ... 2019-05-31 19:48:31 INFO ::MAIN: The /root/nzbget/scripts/nzbToSickBeard.py script completed successfully. 2019-05-31 19:48:46 INFO ::MAIN: Python v2.7 will reach end of life in 215 days. 2019-05-31 19:48:46 WARNING ::MAIN: Please upgrade to a more recent Python version. 2019-05-31 19:48:46 INFO ::MAIN: Checking database structure... 2019-05-31 19:48:46 INFO ::MAIN: Checking if git needs an update 2019-05-31 19:48:46 INFO ::MAIN: No update needed 2019-05-31 19:48:46 INFO ::MAIN: nzbToMedia Version:0db7c3e10c01cee85814a31fd3c5232b1d7d144e Branch:master (Linux 4.15.0-50-generic) 2019-05-31 19:48:46 INFO ::MAIN: ######################################################### 2019-05-31 19:48:46 INFO ::MAIN: ## ..::[nzbToMedia.py]::.. ## 2019-05-31 19:48:46 INFO ::MAIN: ######################################################### 2019-05-31 19:48:46 INFO ::MAIN: Script triggered from NZBGet Version 21.0. 2019-05-31 19:48:47 INFO ::MAIN: Auto-detected SECTION:SickBeard 2019-05-31 19:48:47 INFO ::MAIN: Calling SickBeard:TV to post-process:Last.Week.Tonight.with.John.Oliver.S06E12.720p.AMZN.WEB-DL.DDP2.0.H.26.4-monkee 2019-05-31 19:48:47 INFO ::MAIN: Attempting to auto-detect TV fork 2019-05-31 19:48:47 INFO ::MAIN: SickBeard:TV fork auto-detection successful ... 2019-05-31 19:48:47 INFO ::MAIN: SickBeard:TV fork set to Medusa-api 2019-05-31 19:48:47 WARNING ::SICKBEARD: No media files found in directory /root/nzbget/downloads/completed/TV/Last.Week.Tonight.with.John.Oliver.S06E12.720p.AMZN.WEB-DL.DDP2.0.H.26.4-monkee. Processing this as a failed download 2019-05-31 19:48:47 POSTPROCESS::SICKBEARD: FAILED: The download failed. Sending 'failed' process request to Medusa-api branch 2019-05-31 19:48:47 POSTPROCESS::SICKBEARD: Deleting failed files and folder /root/nzbget/downloads/completed/TV/Last.Week.Tonight.with.John.Oliver.S06E12.720p.AMZN.WEB-DL.DDP2.0.H.26.4-monkee 2019-05-31 19:48:47 INFO ::MAIN: Deleting /root/nzbget/downloads/completed/TV/Last.Week.Tonight.with.John.Oliver.S06E12.720p.AMZN.WEB-DL.DDP2.0.H.26.4-monkee 2019-05-31 19:48:47 ERROR ::MAIN: Unable to delete folder /root/nzbget/downloads/completed/TV/Last.Week.Tonight.with.John.Oliver.S06E12.720p.AMZN.WEB-DL.DDP2.0.H.26.4-monkee 2019-05-31 19:48:47 INFO ::CLEANDIR: Directory /root/nzbget/downloads/completed/TV/Last.Week.Tonight.with.John.Oliver.S06E12.720p.AMZN.WEB-DL.DDP2.0.H.26.4-monkee has been processed and removed ... 2019-05-31 19:48:47 INFO ::MAIN: The /root/nzbget/scripts/nzbToSickBeard.py script completed successfully. 2019-05-31 19:49:02 INFO ::MAIN: Python v2.7 will reach end of life in 215 days. 2019-05-31 19:49:02 WARNING ::MAIN: Please upgrade to a more recent Python version. 2019-05-31 19:49:02 INFO ::MAIN: Checking database structure...

[clinton-hall]

I don't think this is an issue with URL vs .nzb files. Medusa should be passing them all as URL, but once the URL is parsed by NZBGet this should load the nzb.

The issue I have seen myself is that sometimes this URL is passed to NZBGet before the nzb has been loaded on your usenet server, and therefore it fails in the "URL" stage. In NZBGet if you re-download these downloads, they should work (or at least get to the nzb state).

I believe there are settings in Medusa, and NZBGet to wait some time (minimum age) e.g. NZBGet, Settings, Download Queue, PropagationDelay The option sets minimum post age for nzb-files. Very recent files are not downloaded to avoid download failures. The files remain on hold in the download queue until the propagation delay expires, after that they are downloaded.

As far as these not appearing in the logs. If it fails in this URL state, it doesn't trigger the postprocessing scripts, so there is not much I can do within the script side of it.

[x9906789x]

Thanks for the reply. Looking into the URL failure issue I figured out why they were failing. I had added NZBGeek as a search provider in medusa to test it for a trial period and forgot about it. Apparently Medusa is still pulling from NZBGeek sometimes and when NZBGet tries to hit the URL it gets rejected by NZBGeek since i was on a temporary trial account. At that point it fails out but does nothing. I guess that makes this a NZBGet issue as i would think you would want some process to occur if downloads fail at pulling the NZB file down instead of just marking it failed and leaving the show in limbo. This requires manually checking medusa periodically to see if file status has been listed as snatched for too long such that a failure must have occurred but just not been reported back to Medusa.