webdriver icon indicating copy to clipboard operation
webdriver copied to clipboard

Allow Add Cookie and Get All Cookies to work for other site origins/mismatching domains

Open nddipiazza opened this issue 6 years ago • 16 comments

The webdriver spec https://w3c.github.io/webdriver/webdriver-spec.html#add-cookie forces the current browser session to be on the domain where you are adding the cookie to.

This makes tons of sense and I agree with it.

This unfortunately prevents 2 key use cases:

  1. You want to re-use cookies from another webdriver session in a new session to avoid repeating the work it took to get the cookies. Such as having to repeat a login.
  2. Allow a webdriver pool to be shared amongst unrelated threads where each thread might have their own cookies.

The only current work around is to attempt to force the webdriver to go to a 404 error page with something like: webdriver.get("http://the-cookie-domain.com/404adsfasdf"). This would cause the page to go to domain and would allow you to add cookies with addCookie. But this hardly ever works. Because the page is very often protected by SSO, any attempt to go to http://the-cookie-domain.com/* sends you to an SSO login page e.g. http://login.ssodomain.com and now you have the same problem.

We should add some sort of way to do this to the spec.

Ideas how to accomplish this

  1. Change the spec to allow valid cookies to be added a web driver session at any point after creation.

  2. Add a new method webdriver.goToDomain(domain) to allow this use-case. This method should simulate a 200 or 404 HTTP status code response from domain in the browser.

  3. Overload to addCookie could allow this, for example: void addCookie(Cookie var1, String goToDomainBeforeAdding); where the web driver will simulate a 200 or 404 HTTP status code from a get request prior to adding the cookie.

  4. Allow web driver implementation providers to have an Environment variable that can override the cookie domain checks. For example, Firefox could have environment variable NO_COOKIEDOMAIN_VALIDATION=true that would disable these validations https://gist.github.com/nddipiazza/1c8cc5ec8dd804f735f772c038483401

Effect on WebDriver Implementations

Google Chrome

Currently Google Chrome will allow you to add a cookie to the web driver only if you are on some sort of http web page. If you are at the data:,, not found or some other non-web page it will fail with cannot set cookies.

So if you webDriver.get("https://www.google.com"), you can then addCookie of a cookie from any domain you want.

So currently in Google Chrome, you have an (albeit annoying) workaround that works 100% of the time.

Firefox

Currently FireFox will not allow you to add cookies of a domain unless you are actually on that domain, as described in the issue.

GhostDriver (PhantomJS)

Same thing as FireFox. Will need a hack to allow cookies from another domain.

JBrowserDriver

Currently JBrowserDriver does not impose any sort of restriction on where you can add cookies.

nddipiazza avatar Mar 07 '18 20:03 nddipiazza

Hacky workaround for Firefox - you can make the following change and rebuild firefox https://gist.github.com/nddipiazza/1c8cc5ec8dd804f735f772c038483401

This will prevent the validation errors when saving cookies from another domain.

nddipiazza avatar Mar 08 '18 15:03 nddipiazza

Of all the suggested amendments to the spec, only the first ("Change the spec to allow valid cookies to be added a web driver session at any point after creation.") seems reasonable. If that is in place, the other 3 suggestions are not required.

One of the original guiding principles for Selenium WebDriver (which the spec is derived from) is to be sympathetic to a JS implementation, which is why the cookie domain restriction is in place. The same restriction means that you can't use the "Add Cookie" methods to set secure or HTTP-only cookies.

We'd need input from the browser vendors before making any changes to the spec. I would imagine that we'd need to carefully consider the security implications of this carefully. Anything that uses a user-wide shared cookie jar would probably find this problematic to implement safely.

shs96c avatar Mar 13 '18 17:03 shs96c

Hello, I have the same problem, when I am hacking and my target is mywebsite.domain.com, I first apply mywebsite.domain.com/404, some sites make a redirect to login.domain.com. lol I can not add a cookie in the domain: mywebsite.domain.com.

PS: I am using C# with ChromeDriver Eg. chromeDriver.Manage().Cookies.AddCookie(new Cookie(name, value, domain, path, expiry));

reslley avatar Apr 03 '18 22:04 reslley

You want to re-use cookies from another webdriver session in a new session to avoid repeating the work it took to get the cookies. Such as having to repeat a login.

This is exactly my use case. I'm writing an automation bot and I don't want the website to think I'm doing malicious activity. But visiting a page without cookies and then revisiting it with authorized cookies without having filled a login form is something that could raise eyebrows in the website's security logs.

pouriap avatar May 31 '18 08:05 pouriap

I think this also applies to GetAllCookies. Specifically, in my Rust WebDriver client implementation, I have this ugly hack for extracting the cookies for a user-provided URL from the current session so that they can be used in a new one. It's not pretty, and runs into issues with things like SSO. I also haven't been able to find a better workaround than the "use a sub-path that's unlikely to exist" trick.

EDIT: @andreastt which is all to say that I think we should add the Get All Cookies label and maybe update the issue title :)

jonhoo avatar Dec 05 '18 15:12 jonhoo

I use Tshark/Wireshark and Selenium/Firefox together and having to first request a page before being able to add a cookie increases the parsing difficulty of tshark output since I need to ignore the first dummy request. It's not as simple as just waiting till after the dummy request to start tshark captures because of TLS contexts that need to be known from the get go to be able to decrypt the session. Also redirects can add additional complexity.

Being able to set cookies for the webdriver prior to the first page request would greatly simplify things for my use case.

The hack in Firefox w/o having to rebuild is to simply extract the omni.ja file and edit the marionette driver.js and cookie.js files to comment out the relevant checks and then archive the omni.ja back up and restart Firefox. This unfortunately needs to be done though every time Firefox updates.

I do hope in the future this will be allowable by default.

Wallboy avatar Mar 29 '19 06:03 Wallboy

Currently Google Chrome will allow you to add a cookie to the web driver only if you are on some sort of http web page. If you are at the data:,, not found or some other non-web page it will fail with cannot set cookies.

So if you webDriver.get("https://www.google.com"), you can then addCookie of a cookie from any domain you want.

So currently in Google Chrome, you have an (albeit annoying) workaround that works 100% of the time.

OK, it's works fine. But how can I do same when I use proxy (bmp) with my browser? Since I set the proxy settings at the time of start selenium/webdriver session, I cannot set this proxy settings after, it is prohibited.

s-belichenko avatar Apr 04 '19 09:04 s-belichenko

Currently Google Chrome will allow you to add a cookie to the web driver only if you are on some sort of http web page. If you are at the data:,, not found or some other non-web page it will fail with cannot set cookies. So if you webDriver.get("https://www.google.com"), you can then addCookie of a cookie from any domain you want. So currently in Google Chrome, you have an (albeit annoying) workaround that works 100% of the time.

OK, it's works fine. But how can I do same when I use proxy (bmp) with my browser? Since I set the proxy settings at the time of start selenium/webdriver session, I cannot set this proxy settings after, it is prohibited.

Hmm, All works fine, my fault.

s-belichenko avatar Apr 04 '19 14:04 s-belichenko

My webfoo isnt that strong (I am an embedded/sysprog guy) but I don't see how this limitation can be any practical when a get request on a session in any non-trivial case loads not only URLs from the given domain but several more by elements from that initial page (via JS etc). To set cookies for all domains properly the loading would have to be strictly done in lockstep under the control of the caller. This is particularly annoying for subdomains which are often used together in websites but of course not limited to them.

Most probably I am overlooking something or the library I am using (Python binding of selenium) does not map perfectly the behavior intended by the spec. Can someone please explain how users of the API are supposed to create cookies for different domains loaded in a single call (the 404 hack above does not even remotely assemble something I would call thought through TBH :)

stefanct avatar Jun 01 '20 10:06 stefanct

Are there any plans on implementing this?

mszajowskiwr avatar Apr 01 '21 09:04 mszajowskiwr

Is the workaround still working on Chrome? webDriver.get("https://www.google.com") Opening above url and then addCookie gives me org.openqa.selenium.InvalidCookieDomainException: invalid cookie domain

Please could somebody help with the workaround? I am blocked because Chromedriver was fixed recently and I am unable to addCookie which we used to login to app. Before login its on domain1 and then after login it goes to domain2. I used to open domain1, set cookies and reload and I used to land on domain2. Any idea how can I achieve this? I do not want to login through UI every time for test automation.

srilakshmishankar avatar May 10 '21 13:05 srilakshmishankar

Copy from #1593, scenario where add cookie for 'www.google.com' won't work:

The Add Cookie method only allows setting a cookie with a domain equal to the current browsing context's document. This means that if a domain always redirects cross-origin, there's no way to set a cookie on the domain.

For example, let's say a company has an internal site acme-corp.com, and accessing any page on acme-corp.com results in a cross-origin redirect to login.acme.com. It becomes impossible to set any cookie with domain acme-corp.com.

Can this be changed so that any domain in the current browsing context's redirect chain is valid?

(For context, this issue came up after Chrome tightened up enforcement of the domain matching rule. We started getting bug reports about Add Cookie failing due to the cookie domain mismatching, in particular when adding a cookie on domain A immediately after navigating to a page that does an A->B cross-origin redirect.)

Can webdriver implementation help with the above? It's blocking test, which the only work around currently is testing against an older chrome browser version.

amaranthjinn avatar May 25 '21 20:05 amaranthjinn

Hello

I am seeing this issue on Selenium 4.1 running on Edge 99 (test code in C#)... Our framework was running only for Chrome and Firefox. We decided to add support for Edge. The browser opens, and the url is visited correctly, but I get the error. For the rest of browsers I don't have issues. Anyone knows how to solve it?

Thanks

jprealini avatar Mar 31 '22 21:03 jprealini

Hi,

This issue is originally opened more than 4 years ago, and it seems to me like there is still no solution. Is there any news/changes? What's the status? This behavior makes setting cookie impossible if your whole domain is behind SSO on some other domain.

matijakrajnik avatar Nov 21 '22 10:11 matijakrajnik

Yeah, the behavior is by design, aka, not going to change.

amaranthjinn avatar Nov 21 '22 19:11 amaranthjinn

Puppeteer has no such limitation.

ghost avatar Nov 26 '23 04:11 ghost