Skill-Share-Crawler---DL icon indicating copy to clipboard operation
Skill-Share-Crawler---DL copied to clipboard

Can't reliably download larger courses

Open MileyHollenberg opened this issue 6 years ago • 7 comments

Whenever I download a course with 30 or more videos in it I have a really high change of this tool crashing because one of the requests for the video file got timed out, there seems to be no method of re-trying a failed download and the code expects only the happy flow.

Please implement a method of re-trying a video download instead of letting the tool crash (if this happens about 3 times or so I'm also locked out of signing in to my account for about 24 hours, this means that I can only download a couple of my courses for safe-keeping before I need to wait a day to try again)

MileyHollenberg avatar Feb 28 '18 08:02 MileyHollenberg

Ola @zezioen,

In the past few days I've been working hard and I ended up running out of time to respond, sorry.

Recently this project is stopped and is becoming obsolete, mainly with the written logics. I have in mind to rewrite all the code for better syntax and logic, when I performed the videos downloas a long time ago, I had no problems with this video and ban approach for 24 hours. I'll look over the weekend for a solution to the case.

If you write some solution please send us a push for the project. Thank you.

tharyckgusmao avatar Mar 08 '18 02:03 tharyckgusmao

No problem, I tried to change it myself as well but my NodeJS experience is quite rusty so I'm not sure how to fix it quickly (plus I get banned for 24 hours after about 3 attempts which doesn't help). Would be cool if you got this to work this weekend :)

MileyHollenberg avatar Mar 08 '18 08:03 MileyHollenberg

Okay, I'll implement a solution, just wait a little bit.

Do not forget to leave that little star to strengthen the project. Thank you

tharyckgusmao avatar Mar 08 '18 13:03 tharyckgusmao

I'm wondering if this is what happened to me. I've tried 3 times so far to download the course 2036367805 and each time the program crashes

` What do you want to do ? 1 - Download Videos per ID ? Enter ID video Skill Share: 2036367805

Video Music Theory Comprehensive: Part 1Progress

Video #0 [-------------------] 0% Video #1 [-------------------] 0% Video #2 [-------------------] 0% Video #3 [-------------------] 0% Video #4 [-------------------] 0% Video #5 [-------------------] 0% Video #6 [-------------------] 0% Video #7 [-------------------] 0% Video #8 [-------------------] 0% Video #9 [-------------------] 0% Video #10 [-------------------] 0% Video #11 [-------------------] 0% Video #12 [-------------------] 0% Video #13 [-------------------] 0% Video #14 [-------------------] 0% Video #15 [-------------------] 0% Video #16 [-------------------] 0% Video #17 [-------------------] 0% Video #18 [-------------------] 0% Video #19 [-------------------] 0% Video #20 [-------------------] 0% Video #21 [-------------------] 0% Video #22 [-------------------] 0% Video #23 [-------------------] 0% Video #24 [-------------------] 0% Video #25 [-------------------] 0% Video #26 [-------------------] 0% Video #27 [-------------------] 0% Video #28 [-------------------] 0% Video #29 [-------------------] 0% Video #30 [-------------------] 0% Video #31 [-------------------] 0% Video #32 [-------------------] 0% Video #33 [-------------------] 0% Video #34 [-------------------] 0%

undefined:1 undefined ^

SyntaxError: Unexpected token u in JSON at position 0 at Object.parse (native) at Request.request [as _callback] (/Users/rob/Programming/Projects/Others/Skill-Share-Crawler---DL/app/components/DlPerId.js:61:30) at self.callback (/Users/rob/Programming/Projects/Others/Skill-Share-Crawler---DL/node_modules/request/request.js:186:22) at emitOne (events.js:96:13) at Request.emit (events.js:188:7) at Request.onRequestError (/Users/rob/Programming/Projects/Others/Skill-Share-Crawler---DL/node_modules/request/request.js:878:8) at emitOne (events.js:96:13) at ClientRequest.emit (events.js:188:7) at TLSSocket.socketErrorListener (_http_client.js:310:9) at emitOne (events.js:96:13)

npm ERR! Darwin 17.4.0 npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "run" "start" npm ERR! node v6.9.1 npm ERR! npm v3.10.8 npm ERR! code ELIFECYCLE npm ERR! [email protected] start: node index.js npm ERR! Exit status 1 npm ERR! npm ERR! Failed at the [email protected] start script 'node index.js'. npm ERR! Make sure you have the latest version of node.js and npm installed. npm ERR! If you do, this is most likely a problem with the skillshare_dl package, npm ERR! not with npm itself. npm ERR! Tell the author that this fails on your system: npm ERR! node index.js npm ERR! You can get information on how to open an issue for this project with: npm ERR! npm bugs skillshare_dl npm ERR! Or if that isn't available, you can get their info via: npm ERR! npm owner ls skillshare_dl npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request: npm ERR! /Users/rob/Programming/Projects/Others/Skill-Share-Crawler---DL/npm-debug.log`

tarasis avatar Mar 18 '18 15:03 tarasis

To add to the info, when I try to download a larger course- in this case the course https://www.skillshare.com/classes/Become-a-Great-Singer-Your-Complete-Vocal-Training-System/1213366843 - I receive the error:

skillshare-crawler-dl/app/components/DlPerId.js:90
            .filter(s => s.avg_bitrate)
             ^

TypeError: Cannot read property 'filter' of undefined

shpshft avatar Jul 03 '18 05:07 shpshft

probably a video with different structure or escaping the error

{error_code: "VIDEO_NOT_PLAYABLE"}

currently this code only works for the standards that it believes is right, it is a little old fashioned not dealing with these errors.

tharyckgusmao avatar Aug 31 '18 13:08 tharyckgusmao

I wonder if its possible to console.log or fetch all the video download links so one could simply export them to jdownloader...It would be awesome...i looked at the code and found the code does something with the hashed links from api but have no idea..(iam a NooB)..if anybody could help...

mayursaroj avatar Feb 09 '19 17:02 mayursaroj