httpx icon indicating copy to clipboard operation
httpx copied to clipboard

Fix URL params to merge with existing query parameters instead of replacing them

Open ferstar opened this issue 5 months ago • 0 comments

Summary

This PR fixes a long-standing issue where URL(url, params=params) and Request(method, url, params=params) would completely replace existing query parameters in the URL instead of merging them. This behavior was inconsistent with the Python requests library and unintuitive for users.

Problem

Previously, when creating a URL or Request with additional parameters:

# Before this fix
url = httpx.URL("https://api.example.com?token=abc123", params={"page": "1"})
print(url)  # https://api.example.com?page=1  (token=abc123 was lost!)

request = httpx.Request("GET", "https://api.example.com?token=abc123", params={"page": "1"})
print(request.url)  # https://api.example.com?page=1  (token=abc123 was lost!)

The original query parameters (token=abc123) were completely discarded.

Solution

After this fix, parameters are intelligently merged:

# After this fix
url = httpx.URL("https://api.example.com?token=abc123", params={"page": "1"})
print(url)  # https://api.example.com?token=abc123&page=1  (parameters merged!)

request = httpx.Request("GET", "https://api.example.com?token=abc123", params={"page": "1"})
print(request.url)  # https://api.example.com?token=abc123&page=1  (parameters merged!)

Behavior Details

The new logic handles various cases intelligently:

  1. Non-empty params: Merged with existing query parameters

    URL("https://example.com?a=1", params={"b": "2"})
    # Result: "https://example.com?a=1&b=2"
    
  2. Empty dict params: Preserves existing query parameters

    URL("https://example.com?a=1", params={})
    # Result: "https://example.com?a=1"
    
  3. None params: Preserves existing query parameters

    URL("https://example.com?a=1", params=None)
    # Result: "https://example.com?a=1"
    
  4. Overlapping parameter names: New values override old ones

    URL("https://example.com?a=old", params={"a": "new", "b": "2"})
    # Result: "https://example.com?a=new&b=2"
    
  5. QueryParams objects: Used directly (preserves existing copy_* method behavior)

    url = URL("https://example.com?a=1&b=2")
    url.copy_remove_param("a")  # Still works as expected
    # Result: "https://example.com?b=2"
    

Compatibility

  • ✅ Backwards compatible: All existing copy_* methods work unchanged
  • ✅ Test coverage: All existing tests pass with updated expectations
  • ✅ Consistent with requests: Matches Python requests library behavior
  • ✅ Intuitive: Behavior now matches user expectations

Files Changed

  • httpx/_urls.py: Updated URL.init params handling logic
  • tests/models/test_url.py: Updated test expectations for merge behavior
  • tests/models/test_requests.py: Updated test expectations for merge behavior

Testing

Added comprehensive tests covering:

  • Basic parameter merging
  • Edge cases (empty params, None params, overlapping names)
  • Request class integration
  • Backwards compatibility with copy_* methods
  • URL object as input parameter

All existing tests pass with the new behavior.

Related Issues

Fixes #652 - params overrides query string in url

Breaking Changes

This is technically a breaking change in behavior, but it fixes unintuitive behavior that was likely causing bugs in user code. The new behavior is:

  1. More intuitive and matches user expectations
  2. Consistent with the popular requests library
  3. Better handles real-world use cases

Users who were relying on the old replacement behavior can achieve the same result by manually constructing URLs without existing query parameters.

ferstar avatar Aug 01 '25 04:08 ferstar