Chris Lyon

That works. I basically just needed an onload callback so that I could set a custom anonymous ID with `window.analytics.user().anonymousId(foo)` once their script loaded.

I've figured out a working solution to this issue that fits my needs, but is a bit involved (involving the need for an async driver pool). If this project is...

@AndreuJove The gist of it is that because `process_request()` can either return a standard response object or a [twisted deferred](https://twistedmatrix.com/documents/16.4.1/core/howto/defer-intro.html) (and because scrapy is itself built on twisted), the handling...

Unfortunately I don't yet. The code I have is private (and is coupled to private libraries). I'd be down to fork and integrate my changes when I get some free...

@vbuterin2 I think your question is a duplicate of https://github.com/clemfromspace/scrapy-selenium/issues/76

I've run into this issue before. There's a couple of ways to handle this depending on if you're using Chrome or Firefox, but I found this article pretty helpful https://piprogramming.org/articles/How-to-make-Selenium-undetectable-and-stealth--7-Ways-to-hide-your-Bot-Automation-from-Detection-0000000017.html...

@Mathoholic You've hit something that's unfortunately pretty common nowadays, and isn't just limited to user agents. There's no easy answer to this, because bot detection and countermeasures are an evolving...

@wondering639 There's only a single browser instance created for the lifecycle of the downloader middleware. I suppose all you'd need to do is avoid running this statement when your crawl...

I've got the same concerns. Thanks for taking the time to write this, by the way @clemfromspace! It's been super helpful! I think Selenium + Scrapy is a great combination.

I also agree with this issue, and have seen it abused in some arguments (such as [this one](http://en.arguman.org/react-is-mostly-hype/21836)). As it is now, the buck stops at a reported fallacy with...