Lazy loading issues with embed tweets on Medium

[bansan85]

Describe the bug Twitter and Github embedded widget are not saved when deferred.

To Reproduce Steps to reproduce the behavior:

1. Go to https://medium.com/androidmood/comprendre-larchitecture-mvvm-sur-android-aa285e4fe9dd
2. Don't scroll
3. Download page with SingleFile
4. Missing twitter and github embedded widget

If I scroll the whole page before save, nothing is missing.

I also tried to tick the "scroll" and "zooming out the page" option without success.

But it's strange that "zooming out the page" doesn't work because I can clearly see that Twitter and Github embedded widgets are loading during this step. So I need to download twice the page. The first time to load data in the browser, the second time to really save it on the html page generated by SingleFile.

Screenshots

Original page (good)

Singlefile page (bad)

Environment

• OS: Win10 Pro
• Browser: Chrome AND Firefox
• Version: 64

[gildas-lormeau]

Medium implements a complicated mechanism to lazy load contents. Unfortunately, you have to scroll to save the page properly.

[gildas-lormeau]

I was able to (almost) fix the issue. Now SingleFile can save all the frames (without scrolling) except the embed tweet, unfortunately. I also added an option to ensure deferred frames are loaded before being saved. The fix and the new option will be available in the next version.

I keep the issue open because of the embed tweet bug.

[gildas-lormeau]

For the record, enabling Images > save deferred images > zoom out the page fixes the issue with the embed tweet.

[bansan85]

Thanks. I confirm that enabling "load deferred frames" option solves the problem about the missing image. Great job.

[gildas-lormeau]

Thank you for the feedback. Actually, you don't even need to enable the option but it's safer to do so.

[bansan85]

I didn't tested without. But now I have.

I confirm : I need to have "load deferred frames" to have Twitter AND Github frames.

Without "load deferred frames", I have none of them.

[gildas-lormeau]

Thank you for the information. I was referring to Medium actually, I didn't really test the fix/new option on other websites.

[bansan85]

Me too, with the link from the first message... Strange...

Edit : Strange again. A just made a new test, I lost the tweet frame but still have the Github frame...

Edit 2 : Tweet frame came back and doesn't disappear anymore ⁉️

[gildas-lormeau]

Sorry for the misunderstanding regarding the website you're testing. Medium sucks, a lot. They use over engineered implementations to handle lazy loaded contents. Moreover, they don't use the same technique to handle the lazy-loading on GitHub and Twitter frames. That's probably why you see strange results. Note that lazy-loaded content management remains in best-effort mode in SingleFile. Unfortunately, I can't guarantee that it will work in 100% of cases.