devdocs
devdocs copied to clipboard
[low priority refactoring] generalize executing JS during scraping
If doc sites use client-side JavaScript to produce or fetch some of their content, we need to launch a headless browser and execute it. Currently scraper subclasses do that individually using capybara, e.g. https://github.com/freeCodeCamp/devdocs/blob/master/lib/docs/scrapers/angularjs.rb#L42-L45 but it might be better to extract a helper method or config option, a little more like this: https://github.com/freeCodeCamp/devdocs/blob/master/lib/docs/scrapers/webpack.rb#L43-L46
cc @dangeranger
We only have three legacy docs requiring capybara: https://devdocs.io/angularjs~1.8/, https://devdocs.io/angular~2/, https://devdocs.io/immutable/
Thus, the refactoring is indeed very low priority.