When the URL contains request parameters and the `params` parameter is set, the request parameters in the URL will disappear unexpectedly.
The starting point for issues should usually be a discussion...
https://github.com/encode/httpx/discussions
Possible bugs may be raised as a "Potential Issue" discussion, feature requests may be raised as an "Ideas" discussion. We can then determine if the discussion needs to be escalated into an "Issue" or not.
This will help us ensure that the "Issues" list properly reflects ongoing or needed work on the project.
- [ ] Initially raised as discussion #...
As shown in the following Python code, my expected output should be to reasonably concatenate the request parameters in params with those in the url. requests handled this part well, but httpx gave the following unexpected output:
import requests
import httpx
url = 'https://safebooru.org/index.php?page=post&s=list'
params = {
'pid': 0,
'tags': 'k-on!',
}
response = requests.get(url=url, params=params)
print(response.request.url) # As expected: https://safebooru.org/index.php?page=post&s=list&pid=0&tags=k-on%21
response = httpx.get(url=url, params=params)
print(response.request.url) # Unexpectedly, the request parameters in the URL were overridden by params: https://safebooru.org/index.php?pid=0&tags=k-on%21
My httpx version is 0.28.1
If you want to avoid params getting dropped/overwritten when a URL already contains a query, you can pre-merge the query yourself with qs_codec (my Python port of the Node.js qs library) and hand httpx a fully-formed URL. This also gives you proper nested param handling (a[b][0]=...) and configurable encoding (spaces as %20 vs +, etc).
Safely merge existing URL query with new params
import httpx
from urllib.parse import urlsplit, urlunsplit
import qs_codec as qs
base_url = "https://api.example.com/search?a=b&c=d" # already has query
new_params = {
"c": "D", # will co-exist or override per your choice below
"e": "f",
"tags": ["x", "y"], # lists -> multiple entries or bracketed, as configured
}
parts = urlsplit(base_url)
# Decode existing query string into a nested dict (robust parsing, preserves duplicates)
existing = query_string.decode(parts.query)
# Merge how you want (examples):
# a) combine duplicates into a list
for k, v in new_params.items():
if k in existing:
# promote to list and append
existing[k] = ([existing[k]] if not isinstance(existing[k], list) else existing[k]) + (v if isinstance(v, list) else [v])
else:
existing[k] = v
# b) OR choose "last-wins" by just updating:
# existing.update(new_params)
# Re-encode exactly once (RFC 3986 = spaces as %20; switch to RFC1738 for '+')
query_string = qs.encode(existing, qs.EncodeOptions(format=qs.Format.RFC3986))
final_url = urlunsplit((parts.scheme, parts.netloc, parts.path, query_string, parts.fragment))
resp = httpx.get(final_url)
Nested query parameters (qs-style)
If the API expects bracketed or indexed keys (a common pain point), qs_codec handles it:
# Build nested structure directly in Python
payload = {
"search": {"term": "billing", "filters": [{"field": "status", "op": "eq", "value": "open"}]},
"page": 2
}
# Encodes to: search[term]=billing&search[filters][0][field]=status&...&page=2
query_string = qs.encode(payload) # percent-encoded by default
url = f"https://api.example.com/tickets?{query_string}"
resp = httpx.get(url)
Encoding edge cases (spaces, %, etc.)
qs_codec lets you pick encoding style and avoids double-encoding:
- RFC 3986 (default) → space =
%20 - RFC 1738 → space =
+
qs.encode({"q": "foo bar"}, qs.EncodeOptions(format=qs.Format.RFC3986)) # q=foo%20bar
qs.encode({"q": "foo bar"}, qs.EncodeOptions(format=qs.Format.RFC1738)) # q=foo+bar
This approach works today without changes to httpx: you bypass the internal params merge (the source of surprises here) and send an already-correct query string.