web_scraper icon indicating copy to clipboard operation
web_scraper copied to clipboard

Mainthread blocking.

Open dvird opened this issue 3 years ago • 3 comments

The library parsing and doing all things on main thread. which slows down the all app

dvird avatar Aug 05 '21 17:08 dvird

Are you using the async functions for all events?

tusharojha avatar Aug 09 '21 15:08 tusharojha

sure. but i need it at scale, so im parsing like 36 pages at the same time and its blocking the thread a little bit.

also the way its working is not best for performance. becasue when the message of the parsing is getting to the event loop so its stucking till all parsing end. its not as chunk mechanism or separate isolate

dvird avatar Aug 11 '21 17:08 dvird

Agreed. It could be a valid improvement to do.

tusharojha avatar Aug 22 '21 14:08 tusharojha