DynamicScraper from the example not doing anything

[jeremythille]

I'm trying out the examples exactly as written. The HackerNews example with scraperjs.StaticScraper works perfectly fine and I get a big array of strings.

However, replacing Static with Dynamic doesn't do anything. At all. The execution just ends with no error. I tried debugging it, adding try/catch statements, but I still get no error. I tried "npm install phantom" too, but it doesn't change anything for scraperjs. I have no idea what's wrong here.

[abinashmeher999]

+1

[skibish]

@jeremythille @abinashmeher999 It will work, if you downgrade phatomjs to 1.9.8 version as mentioned in #53

[QuentinRoy]

I downgraded to phantomjs 1.9.8 but it still does not work on Mac OS 10.11.5.

[Nosion]

Removing the folder node_modules and install again did the trick for me.

[brennanerbezniksc]

Downgraded to 1.9.8 and no luck

[coopernurse]

Check your version of node. It looks like DynamicScraper doesn't work with newer node versions. node 7.9 didn't work for me, but 4.8.2 worked fine.

I'm building a docker image to encapsulate the correct version of node/phantom/scraperjs, as it appears there's a pretty specific set of versions that are required to make this all work.

This Dockerfile worked for me:

FROM node:4.8

WORKDIR /scraperjs
RUN npm install scraperjs
RUN npm install -g '[email protected]'
ENV NODE_PATH=/scraperjs/node_modules

then:

# assumes dynamic HN example is in 'hn.js'
$ sudo docker build -t coopernurse/scraperjs .
$ sudo docker run --rm -v `pwd`:/code coopernurse/scraperjs node /code/hn.js