PoliWebex
PoliWebex copied to clipboard
sup3rgiu
-r retry option
Sometimes the download fails due to erratic errors, especially when downloading a large number of vids from a list, it happens often that 2-3 videos are not downloaded.
At the end of execution poliwebex reports the videos that have not been downloaded.
Would it be complicated to add a retry feature?
Something like: -r, --retry [number] that after has downloaded the videos in the list, retries a number of times the failed videos before terminating the execution.
Ciao @sup3rgiu I tried to implement this feature by myself, even if i know almost nothing of either javascript and node. However, in my head my attempt should work: https://github.com/valerionew/PoliWebex/commit/b9857a037c92f32e31fc134003f8bf48c5bba968 Instead, it does not. Instead of sequentially executing .
Do you have any suggestion? Am i doing something wrong?
This is the console output:
PS C:\Users\valer\Documents\PoliWebex> node poliwebex -r 10 -f "test.txt" -o "test/"
Project powered by @sup3rgiu
Features: PoliMi Autologin - Multithreading download
Using aria2 version 1.35.0
Using ffmpeg version git-2020-05-04-5767a2e Copyright (c) 2000-2020 the FFmpeg developers
Video URLs: [
'https://politecnicomilano.webex.com/recordingservice/sites/politecnicomilano/recording/b0d261950c9544ee8747d4d1d3e6bd12',
'https://politecnicomilano.webex.com/recordingservice/sites/politecnicomilano/recording/8c6d9d0510604eb9fg9f2c9fd84f3105',
'https://politecnicomilano.webex.com/recordingservice/sites/politecnicomilano/recording/62557ada28c7496780bab0d51af85cbd'
]
Output Directory: test/
Reusing password saved in system's keychain!
Launching headless Chrome to perform the OpenID Connect dance...
Reusing password saved in system's keychain!
Launching headless Chrome to perform the OpenID Connect dance...
Reusing password saved in system's keychain!
Launching headless Chrome to perform the OpenID Connect dance...
Reusing password saved in system's keychain!
Launching headless Chrome to perform the OpenID Connect dance...
Reusing password saved in system's keychain!
Launching headless Chrome to perform the OpenID Connect dance...
Reusing password saved in system's keychain!
Launching headless Chrome to perform the OpenID Connect dance...
Reusing password saved in system's keychain!
Launching headless Chrome to perform the OpenID Connect dance...
Reusing password saved in system's keychain!
Launching headless Chrome to perform the OpenID Connect dance...
Reusing password saved in system's keychain!
Launching headless Chrome to perform the OpenID Connect dance...
Reusing password saved in system's keychain!
Launching headless Chrome to perform the OpenID Connect dance...
Reusing password saved in system's keychain!
Launching headless Chrome to perform the OpenID Connect dance...
(node:2936) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 exit listeners added to [process]. Use emitter.setMaxListeners() to increase limit
(node:2936) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGINT listeners added to [process]. Use emitter.setMaxListeners() to increase limit
(node:2936) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGTERM listeners added to [process]. Use emitter.setMaxListeners() to increase limit
(node:2936) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit
Navigating to WebEx login page...
Navigating to WebEx login page...
Navigating to WebEx login page...
Navigating to WebEx login page...
Navigating to WebEx login page...
Navigating to WebEx login page...
Navigating to WebEx login page...
Navigating to WebEx login page...
Navigating to WebEx login page...
Navigating to WebEx login page...
Navigating to WebEx login page...
Filling in Servizi Online login form...
Filling in Servizi Online login form...
Filling in Servizi Online login form...
Filling in Servizi Online login form...
Filling in Servizi Online login form...
Filling in Servizi Online login form...
Filling in Servizi Online login form...
Filling in Servizi Online login form...
Filling in Servizi Online login form...
Filling in Servizi Online login form...
Filling in Servizi Online login form...
Ovviamente i link di test sono deliberatamente sbagliati per triggerare il retry
Hi Valerio! I'm really sorry for the late response.
The problem with your solution is that downloadVideo(...) is an async function, so you should put an await.
Here you can find a working solution with some comments:
https://gist.github.com/sup3rgiu/ef381fe1163daab6791fa3186b818c9a
However, PoliWebex already retry to download up to 5 times a failed download (have a look at the segmentedDownload() and directDownload() functions).
So, I'm not sure that another retry option would be useful.
I'm 100% sure it would be useful to retry after some time, because that's what i do by hand every time and it does work.
I'll check out your gist, thanks!
Hi, I would really appreciate a retry function too :) I also retry after some minutes (those videos in a new list without the segmenting function) and it works :) BTW thanks for that list of failed downloads at the end... it's very useful XD
