Download failed with "AttributeError: 'NoneType' object has no attribute 'string'"

[scottgmcleod]

Subject of the issue

Download failed with "AttributeError: 'NoneType' object has no attribute 'string'"

Your environment

• Operating System (name/version): MacOS 10.14.6
• Python version:3.6
• youtube-dl version:2020.03.01
• edx-dl version:0.1.13

Steps to reproduce

Tell us how to reproduce this issue. Please provide us the course URL, and the specific subsection or unit if possible. The tool sees the course I am enrolled in at Stanford but when I try to download the url (https://lagunita.stanford.edu/courses/HumanitiesSciences/StatLearning/Winter2016/course/) it fails.

Expected behaviour

It should commence the download

Actual behaviour

Output from edx-dl -x stanford -u [email protected] https://lagunita.stanford.edu/courses/HumanitiesSciences/StatLearning/Winter2016/course/

edx_dl version 0.1.13 Password: Building initial headers for future requests. Getting initial CSRF token. Found CSRF token. Logging into Open edX site: https://lagunita.stanford.edu/login_ajax Extracting course information from dashboard. Traceback (most recent call last): File "/Users/scottm/miniconda2/bin/edx-dl", line 8, in sys.exit(main()) File "/Users/scottm/miniconda2/lib/python3.6/site-packages/edx_dl/edx_dl.py", line 1028, in main for selected_course in selected_courses} File "/Users/scottm/miniconda2/lib/python3.6/site-packages/edx_dl/edx_dl.py", line 1028, in for selected_course in selected_courses} File "/Users/scottm/miniconda2/lib/python3.6/site-packages/edx_dl/edx_dl.py", line 186, in get_available_sections sections = page_extractor.extract_sections_from_html(page, BASE_URL) File "/Users/scottm/miniconda2/lib/python3.6/site-packages/edx_dl/parsing.py", line 403, in extract_sections_from_html for i, section_soup in enumerate(sections_soup, 1)] File "/Users/scottm/miniconda2/lib/python3.6/site-packages/edx_dl/parsing.py", line 403, in for i, section_soup in enumerate(sections_soup, 1)] File "/Users/scottm/miniconda2/lib/python3.6/site-packages/edx_dl/parsing.py", line 392, in _make_subsections for i, s in enumerate(subsections_soup, 1)] File "/Users/scottm/miniconda2/lib/python3.6/site-packages/edx_dl/parsing.py", line 392, in for i, s in enumerate(subsections_soup, 1)] AttributeError: 'NoneType' object has no attribute 'string'

[Ken0sis]

Exact same issue for me, except I'm on Python 2.7.

[scottgmcleod]

The tool does work for an edX course I took. But not lagunita...

[fixfo]

Same error on lagunita. Ubuntu 19.10, python 3.7.5 (system)

[mohan-barathi]

Same error for me ..! I am using Python 2.7.14 on Windows 10

I can download edx courses ..! But I am facing issue in lagunita.stanford.edu

Is there anyother way to download, as Lagunita is retiring and will shut down at 12 noon Pacific Time on March 31, 2020.

[MATRIX30]

Same error for me ..! I am using Python 2.7.14 on Windows 10

I can download edx courses ..! But I am facing issue in lagunita.stanford.edu

Is there anyother way to download, as Lagunita is retiring and will shut down at 12 noon Pacific Time on March 31, 2020.

Replace your parsing.py with the one found in the zip file below. parsing.zip

[Ken0sis]

MATRIX30's edit of the parsing.py file seems to work for me. Thank you!

[scottgmcleod]

Yep Worked for me too. Thanks MATRIX30!

[ny83427]

"ERROR: This video is no longer available due to a copyright claim by Stanford University. Sorry about that."😭

parsing.py works great but failed before copyright claim:(

[mohan-barathi]

@ny83427 comment out line 42 on utils.py, and add a print statement print (e, "move on..!")

Usually introduction videos are with copyrights .. you can download them manually ..!! 🔨 (quick fix..!)

[ny83427]

@mohan-barathi HOLY COW! This works like a charm. Thank you so much, dude!

[ichit]

Thanks

[ny83427]

@ny83427 comment out line 42 on utils.py, and add a print statement print (e, "move on..!")

Usually introduction videos are with copyrights .. you can download them manually ..!! 🔨 (quick fix..!)

I do think the workflow of the code can be improved a little bit as you suggested, currently, one failure will stop the whole world, which is the worst scearnio. Continue fetching other resources while output an error list would be more friendly.

[RakeshIIITD]

It's not working for me for lagunita. Same string error is coming even after replacing parsing.py as suggested by MATRIX20. ERROR

edx_dl version 0.1.13 Building initial headers for future requests. Getting initial CSRF token. Found CSRF token. Logging into Open edX site: https://lagunita.stanford.edu/login_ajax Extracting course information from dashboard. Traceback (most recent call last): File "/usr/local/bin/edx-dl", line 10, in sys.exit(main()) File "/usr/local/lib/python2.7/dist-packages/edx_dl/edx_dl.py", line 1028, in main for selected_course in selected_courses} File "/usr/local/lib/python2.7/dist-packages/edx_dl/edx_dl.py", line 1028, in for selected_course in selected_courses} File "/usr/local/lib/python2.7/dist-packages/edx_dl/edx_dl.py", line 186, in get_available_sections sections = page_extractor.extract_sections_from_html(page, BASE_URL) File "/usr/local/lib/python2.7/dist-packages/edx_dl/parsing.py", line 403, in extract_sections_from_html for i, section_soup in enumerate(sections_soup, 1)] File "/usr/local/lib/python2.7/dist-packages/edx_dl/parsing.py", line 392, in _make_subsections for i, s in enumerate(subsections_soup, 1)] AttributeError: 'NoneType' object has no attribute 'string'

[ny83427]

@RakeshIIITD Replacing parsing.py is not enough.please check @mohan-barathi 's solution:

comment out line 42 on utils.py, and add a print statement print (e, "move on..!")

This works at least for me.

[Druadach]

thanks

[numlockkey]

None pf these solutions work anymore.

[mohan-barathi]

@numlockkey Because the lagunita.stanford.edu is closed on 31-March-2020. That's the reason we were trying the downloads with workarounds, and didn't concentrate on fixing. 😃