Andrew D Bate

Results 65 comments of Andrew D Bate

@GhbSmwc I have implemented this in my fork. You could try https://github.com/andrewdbate/linkgopher/ if you are interested?

@az0 I would need to fix a bug that I've found due to my changes, and then I could submit a pull request.

@StealthBlackOxide I have implemented this in my fork. You could try https://github.com/andrewdbate/linkgopher/ if you are interested?

Here is another error that I sometimes get: ``` Protocol error: Connection closed. Most likely the page has been closed. URL: Stack: Error: Protocol error: Connection closed. Most likely the...

Would it be possible have the SingleFile CLI automatically retry in case of an error? All of these errors come from Puppeteer. Would it be more reliable to use `--back-end=jsdom`...

I followed your other suggestion, which was to use the `sitemap.xml` so I do not need to use the crawl options. (Also, to followup on my comments about jsdom: It...

Here is another error I sometimes get: ``` Protocol error (Target.closeTarget): Target closed. URL: Stack: Error: Protocol error (Target.closeTarget): Target closed. at /usr/src/app/node_modules/puppeteer-core/lib/cjs/puppeteer/common/Connection.js:71:63 at new Promise () at Connection.send (/usr/src/app/node_modules/puppeteer-core/lib/cjs/puppeteer/common/Connection.js:70:16)...

(I think we commented at the same time.) Is playwright more reliable in your experience? Can I use `--crawl-save-session` even though I do not want to crawl? (I prefer to...

I thought I'd try out the `--crawl-save-session` option with `--urls-file` to see how it works. I used an URLs file with ~3800 URLs. I did run out of memory at...