scala-scraper icon indicating copy to clipboard operation
scala-scraper copied to clipboard

Published 20 hours ago • verified publisher icon ruippeixotog

Waiting for real final rendering

Open brunierterry opened this issue 4 years ago • 1 comments

@ruippeixotog You did a great job with this scraper!

Is there a way to extract the content that a page would get asynchronously after the it finished the natural rendering?

brunierterry avatar May 29 '21 10:05 brunierterry

Hi @brunierterry, thanks! For that I think you'd need to rely on HtmlUnit-specific features, of which I'm not 100% familiar. Looking at https://htmlunit.sourceforge.io/javascript-howto.html it seems that HtmlUnit has support for common event handlers, so maybe you can use onload?

ruippeixotog avatar May 30 '21 21:05 ruippeixotog

Closing due to inactivity. Feel free to reopen if you still need support.

ruippeixotog avatar Apr 16 '23 22:04 ruippeixotog