Squidwarc
Squidwarc copied to clipboard
Setting mode in conf.json is ignored
Are you submitting a bug report or a feature request?
Bug report.
What is the current behavior?
Changing the mode from page-only to page-same-domain as described in the manual wont change the search behavior.
{ "use": "puppeteer", "headless": true, "script": "./userFns.js", "mode": "page-same-domain", "depth": 5, "seeds": [ "...." ], "warc": { "naming": "url", "append": "true" }, "connect": { "launch": true, "host": "localhost", "port": 9222 }, "crawlControl": { "globalWait": 60000, "inflightIdle": 1000, "numInflight": 2, "navWait": 8000 } }
What is the expected behavior?
The search mode should change.
How to fix.
If I write the search mode under crawlControl it does change. { "use": "puppeteer", "headless": true, "script": "./userFns.js", "seeds": [ "....." ], "warc": { "naming": "url", "append": "true" }, "connect": { "launch": true, "host": "localhost", "port": 9222 }, "crawlControl": { "globalWait": 60000, "inflightIdle": 1000, "numInflight": 2, "navWait": 8000, "mode": "page-same-domain", "depth": 5 } } In the file config.yml the mode is also listed under crawlControl but its not in the manual.
This seems to be a documentation error rather than a bug.