Squidwarc icon indicating copy to clipboard operation
Squidwarc copied to clipboard

Setting mode in conf.json is ignored

Open DerChaot opened this issue 5 years ago • 1 comments

Are you submitting a bug report or a feature request?

Bug report.

What is the current behavior?

Changing the mode from page-only to page-same-domain as described in the manual wont change the search behavior.

{ "use": "puppeteer", "headless": true, "script": "./userFns.js", "mode": "page-same-domain", "depth": 5, "seeds": [ "...." ], "warc": { "naming": "url", "append": "true" }, "connect": { "launch": true, "host": "localhost", "port": 9222 }, "crawlControl": { "globalWait": 60000, "inflightIdle": 1000, "numInflight": 2, "navWait": 8000 } }

What is the expected behavior?

The search mode should change.

How to fix.

If I write the search mode under crawlControl it does change. { "use": "puppeteer", "headless": true, "script": "./userFns.js", "seeds": [ "....." ], "warc": { "naming": "url", "append": "true" }, "connect": { "launch": true, "host": "localhost", "port": 9222 }, "crawlControl": { "globalWait": 60000, "inflightIdle": 1000, "numInflight": 2, "navWait": 8000, "mode": "page-same-domain", "depth": 5 } } In the file config.yml the mode is also listed under crawlControl but its not in the manual.

DerChaot avatar Aug 15 '19 13:08 DerChaot

This seems to be a documentation error rather than a bug.

BubuAnabelas avatar Aug 18 '19 18:08 BubuAnabelas