Inconsistent behavior after redirects when passing cookies directly

[JanPokorny]

I have noticed that cookies passed directly (no session) behave strangely. When passed as the cookie argument, cookies are persisted after all redirects (even cross-domain and when expired explicitly). When passed as the Cookie header, the header is dropped after any redirect.

I came across this behavior when discussing the same issue for the httpx library (https://github.com/encode/httpx/issues/1404). I do not know whether this behavior is intended. If it is, I would be glad for an explanation behind this design choice.

Expected Result

I would expect cookies passed as the cookie argument to work the same way as if a single-use session was used, and the Cookie header to be passed regardless of redirects.

Actual Result

cookie argument cookies are always passed (regardless of being expired or the redirect being cross-domain), while Cookie header cookies are always dropped (even on same-domain redirects).

Reproduction Steps

Server:

import flask
app = flask.Flask(__name__)

@app.route('/r')
def r():
    return "yes" if "test" in flask.request.cookies else "no"

@app.route('/same_domain_redirect')
def same_domain_redirect():
    return flask.redirect("/r", code=302)

@app.route('/cross_domain_redirect')
def cross_domain_redirect():
    return flask.redirect("http://localhost:5000/r", code=302)

@app.route('/same_domain_redirect_expire')
def same_domain_redirect_expire():
    resp = flask.redirect("/r", code=302)
    resp.set_cookie('test', '', expires=0)
    return resp

if __name__ == '__main__':
    app.run()

Client:

import requests

print(requests.get("http://127.0.0.1:5000/same_domain_redirect", cookies={ "test": "test" }).content)
print(requests.get("http://127.0.0.1:5000/cross_domain_redirect", cookies={ "test": "test" }).content)
print(requests.get("http://127.0.0.1:5000/same_domain_redirect_expire", cookies={ "test": "test" }).content)
print(requests.get("http://127.0.0.1:5000/same_domain_redirect", headers={ "Cookie": "test=test" }).content)
print(requests.get("http://127.0.0.1:5000/cross_domain_redirect", headers={ "Cookie": "test=test" }).content)
print(requests.get("http://127.0.0.1:5000/same_domain_redirect_expire", headers={ "Cookie": "test=test" }).content)

System Information

{
  "chardet": {
    "version": "4.0.0"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.10"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.9.1"
  },
  "platform": {
    "release": "10",
    "system": "Windows"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.25.1"
  },
  "system_ssl": {
    "version": "1010107f"
  },
  "urllib3": {
    "version": "1.26.2"
  },
  "using_pyopenssl": false
}

[sigmavirus24]

This is expected behaviour as the cookies argument doesn't provide a domain attribute for us to check. I've discouraged this usage in the past and even started building something to make handling cookies (and then passing that to Requests) more sensible (such that it would be harder to do this here) but it's incomplete

[JanPokorny]

@sigmavirus24 So why is the Cookie header being cleared? I would expect that cookies passed as the cookies argument to have the same behavior as directly passing the Cookie header, whatever that behavior is.

[sigmavirus24]

So it's a semantic difference. If you pass auth=ClassInheritingFromAuthBase we also behave differently from passing headers={"Authorization": "..."}. The behaviour for scrubbing the header provided by the user that contains sensitive information is a security precaution. Unfortunately the cookies behaviour has been cemented since 0.x releases and the cookies parameter is just a short-cut to pain and misery. Luckily, in our understanding, exceedingly few people use it.

[JanPokorny]

@sigmavirus24 I understand. I ask this because I've been affected by an ugly bug that resulted from a different behavior of requests and httpx, and there's currently a discussion happening in https://github.com/encode/httpx/issues/1404 about what should be the "correct" behavior.

[sigmavirus24]

"correct" would be cookies={"foo": "bar"} is great for testing things but an awful pattern for security in general (cookies=SomeCookieJarPleaseGodNotTheStdlib(...)). Cookie: foo=bar headers set on a request should be assumed to be for a single domain and redirects that end up on HTTP (I can see the case for http -> http not removing the header, but it's safer to remove it), or domain (even to a subdomain) need to remove the header because you can't be 100% certain it's intended for any other requests like that.

[dgtlmoon]

What about when using a cookie jar? https://requests.readthedocs.io/en/latest/user/quickstart/#cookies