autoScroll is scrolling too fast -> additional contents are not loaded

[steph-nb]

URL https://www.defacto.expert/

Describe the bug When running the crawl of the URL above with autoscrolll active in browsertrix-crawler, the newly triggered requests (by the scroll step) are not waited to be finished (maybe wait 2s plus until network idle). Only then the next scroll step should be initiated. And so on.

Maybe it is also woth making a separate autoscrollslow.js to allow correctly harvesting such pages?

[edsu]

I tried using --waitUntil networkidle0 to tell puppeteer to wait until all the requests are finished, which didn't seem to help.

When I enabled logging with --logging stats,pywb,behaviors,behaviors-debug I noticed this message on the console:

{"state":{"segments":1},"msg":"Skipping autoscroll, page seems to not be responsive to scrolling events"}

So maybe there does seem to be something about this page that is causing the autoscroll behavior some difficulty?